-- Logs begin at Wed 2022-08-31 04:38:57 UTC, end at Wed 2022-08-31 05:50:08 UTC. -- Aug 31 05:04:39.410426 dv-cinder-852208-14-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 31 05:04:39.419595 dv-cinder-852208-14-iscsi devstack@keystone.service[81588]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 31 05:04:39.421570 dv-cinder-852208-14-iscsi devstack@keystone.service[81588]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 31 05:04:39 2022] *** Aug 31 05:04:39.421570 dv-cinder-852208-14-iscsi devstack@keystone.service[81588]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 31 05:04:39.421570 dv-cinder-852208-14-iscsi devstack@keystone.service[81588]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 31 05:04:39.421570 dv-cinder-852208-14-iscsi devstack@keystone.service[81588]: nodename: dv-cinder-852208-14-iscsi Aug 31 05:04:39.421570 dv-cinder-852208-14-iscsi devstack@keystone.service[81588]: machine: x86_64 Aug 31 05:04:39.421570 dv-cinder-852208-14-iscsi devstack@keystone.service[81588]: clock source: unix Aug 31 05:04:39.421570 dv-cinder-852208-14-iscsi devstack@keystone.service[81588]: pcre jit disabled Aug 31 05:04:39.421570 dv-cinder-852208-14-iscsi devstack@keystone.service[81588]: detected number of CPU cores: 4 Aug 31 05:04:39.421570 dv-cinder-852208-14-iscsi devstack@keystone.service[81588]: current working directory: / Aug 31 05:04:39.421570 dv-cinder-852208-14-iscsi devstack@keystone.service[81588]: detected binary path: /usr/bin/uwsgi-core Aug 31 05:04:39.422407 dv-cinder-852208-14-iscsi devstack@keystone.service[81588]: your processes number limit is 47680 Aug 31 05:04:39.422407 dv-cinder-852208-14-iscsi devstack@keystone.service[81588]: your memory page size is 4096 bytes Aug 31 05:04:39.422407 dv-cinder-852208-14-iscsi devstack@keystone.service[81588]: detected max file descriptor number: 2048 Aug 31 05:04:39.422407 dv-cinder-852208-14-iscsi devstack@keystone.service[81588]: lock engine: pthread robust mutexes Aug 31 05:04:39.422407 dv-cinder-852208-14-iscsi devstack@keystone.service[81588]: thunder lock: enabled Aug 31 05:04:39.422407 dv-cinder-852208-14-iscsi devstack@keystone.service[81588]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 31 05:04:39.422407 dv-cinder-852208-14-iscsi devstack@keystone.service[81588]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 31 05:04:39.460629 dv-cinder-852208-14-iscsi devstack@keystone.service[81588]: Python main interpreter initialized at 0x55a53ed489c0 Aug 31 05:04:39.460629 dv-cinder-852208-14-iscsi devstack@keystone.service[81588]: python threads support enabled Aug 31 05:04:39.460629 dv-cinder-852208-14-iscsi devstack@keystone.service[81588]: your server socket listen backlog is limited to 100 connections Aug 31 05:04:39.460629 dv-cinder-852208-14-iscsi devstack@keystone.service[81588]: your mercy for graceful operations on workers is 90 seconds Aug 31 05:04:39.461017 dv-cinder-852208-14-iscsi devstack@keystone.service[81588]: mapped 403077 bytes (393 KB) for 2 cores Aug 31 05:04:39.461097 dv-cinder-852208-14-iscsi devstack@keystone.service[81588]: *** Operational MODE: preforking *** Aug 31 05:04:39.461325 dv-cinder-852208-14-iscsi devstack@keystone.service[81588]: *** uWSGI is running in multiple interpreter mode *** Aug 31 05:04:39.461325 dv-cinder-852208-14-iscsi devstack@keystone.service[81588]: spawned uWSGI master process (pid: 81588) Aug 31 05:04:39.461529 dv-cinder-852208-14-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 31 05:04:39.466055 dv-cinder-852208-14-iscsi devstack@keystone.service[81588]: spawned uWSGI worker 1 (pid: 81601, cores: 1) Aug 31 05:04:39.466055 dv-cinder-852208-14-iscsi devstack@keystone.service[81588]: spawned uWSGI worker 2 (pid: 81602, cores: 1) Aug 31 05:04:39.466055 dv-cinder-852208-14-iscsi devstack@keystone.service[81588]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 31 05:04:43.697400 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 05:04:43.697400 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: from cryptography.utils import int_from_bytes Aug 31 05:04:43.698189 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 05:04:43.698189 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: from cryptography.utils import int_from_bytes Aug 31 05:04:43.698301 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 05:04:43.698301 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: from cryptography.utils import int_from_bytes Aug 31 05:04:43.698301 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 05:04:43.698301 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: from cryptography.utils import int_from_bytes Aug 31 05:04:43.747343 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 31 05:04:43.747538 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 31 05:04:43.747618 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 31 05:04:43.747689 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 31 05:04:43.747756 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] command line args: [] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 31 05:04:43.747819 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] command line args: [] {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 31 05:04:43.747961 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 31 05:04:43.748086 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 31 05:04:43.748162 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ================================================================================ {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 31 05:04:43.748229 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ================================================================================ {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 31 05:04:43.748426 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] admin_token = **** {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.748562 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] admin_token = **** {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.748649 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] config_dir = [] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.748724 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] config_dir = [] {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.748799 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.748880 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.748950 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] config_source = [] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.749016 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] config_source = [] {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.749086 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] debug = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.749165 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] debug = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.749307 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.749639 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.749639 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.749788 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.749852 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.749909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.750135 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.750230 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.750299 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.750363 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.750526 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] list_limit = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.750610 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] list_limit = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.750676 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] log_config_append = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.750741 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] log_config_append = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.750899 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.750977 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.751035 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] log_dir = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.751090 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] log_dir = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.751229 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] log_file = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.751320 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] log_file = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.751381 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.751444 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.751655 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.751789 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.751922 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.752016 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.752094 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.752224 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.752339 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.752422 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.752551 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.752633 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.752755 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.752835 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.752947 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.753030 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.753154 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.753234 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.753346 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.753419 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.753532 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.753612 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.753737 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.753817 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.753931 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.754045 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.754207 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.754345 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.754439 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.754530 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.754947 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.36/identity {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.755083 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.36/identity {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.755176 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] publish_errors = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.755256 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] publish_errors = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.755333 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.755457 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.755542 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.755614 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.755730 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.755804 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.755921 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.756021 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.756104 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.756176 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.756371 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] standard_threads = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.756493 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] standard_threads = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.756585 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.756667 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.756784 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.756872 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.756943 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.757013 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.757140 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] use_journal = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.757214 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] use_journal = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.757283 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] use_json = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.757354 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] use_json = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.757425 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] use_stderr = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.757494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] use_stderr = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.757573 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] use_syslog = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.757650 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] use_syslog = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.757805 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.757987 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 05:04:43.758099 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.758174 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.758264 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.758344 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.758533 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.758655 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.758744 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.758825 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.758997 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.759130 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.759209 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.759288 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.759454 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.759576 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.759668 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.759742 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.759910 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.760041 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.760123 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.760196 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.760382 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.760509 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.760589 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.760660 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.760836 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.760969 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.761057 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.761136 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.761294 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.761537 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.761658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.761743 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.761834 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.761979 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.762085 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.762284 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.762408 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.762485 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.762648 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.762723 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.762794 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.762972 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.763094 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.763191 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.763266 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.763445 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.763532 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.763688 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.763823 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.763946 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.764058 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.764169 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.764277 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.764389 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.764497 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.764617 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.764727 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.764842 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.764951 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.765059 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.765169 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.765289 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.765397 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.765518 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.765626 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.765734 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.765843 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.765962 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.766117 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.766229 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.766338 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.766446 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.766554 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.766679 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.766797 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.766912 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.767022 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.767130 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.767238 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.767355 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.767465 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.767573 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.767673 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.767780 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.767888 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.768076 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.768195 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.768306 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.768414 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.768522 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.768631 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.768747 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.768864 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.768976 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.769090 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.769204 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.769319 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.769439 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.769549 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.769643 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.769734 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.769825 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.769917 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.770062 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.770181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.770277 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.770368 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.770461 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.770560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.770663 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.770775 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.770884 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.771017 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.771135 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.771247 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.771357 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.771468 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.771569 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.771683 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.771793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.771912 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.772023 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.772166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.772282 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.772392 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.772512 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.772632 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.772750 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.772864 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.772974 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.773084 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.773193 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.773309 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.773425 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.773547 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.773660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.773770 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.773880 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.773991 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.774117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.774240 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.774352 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.774462 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.774571 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.774682 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.774791 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.774909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.775028 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.775139 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.775251 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.775360 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.775469 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.775586 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.775698 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.775809 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.775919 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.776028 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.776138 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.776261 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.776373 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.776483 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.776599 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.776712 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.776822 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.776997 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.777109 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.777225 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.777336 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.777452 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.777563 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.777688 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.777801 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.777911 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.778035 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.778147 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.778267 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.778391 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.778502 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.778612 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.778722 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.778833 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.778943 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.779132 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.779248 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.779364 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.779475 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.779592 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.779712 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.779831 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.779954 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.780067 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.780176 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.780286 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.780396 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.780575 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.780701 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.780814 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.780926 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.781042 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.781154 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.781264 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.781382 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.781501 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.781612 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.781721 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.781837 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.781950 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.782093 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.782214 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.782325 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.782435 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.782546 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.782656 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.782796 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.782921 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] auth.external = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.783043 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] auth.external = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.783235 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.783350 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.783462 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.783581 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.783703 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.783826 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.783939 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] auth.password = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.784058 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] auth.password = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.784170 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] auth.token = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.784281 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] auth.token = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.784392 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.784511 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.784629 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.784741 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.784851 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.784962 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.785072 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.785192 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.785312 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.785424 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.785537 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.785648 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.785648 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.785853 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.785853 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] credential.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.786108 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] credential.caching = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.786223 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.786334 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.786334 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.786553 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.786666 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.786776 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.786887 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.786997 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.787108 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.787231 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.787344 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.787460 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.787571 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.787685 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.787795 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.787913 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.788025 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.788135 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.788247 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.788357 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.788468 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.788586 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.788698 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.788808 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.788918 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.789028 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.789140 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.789258 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.789371 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.789482 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.789593 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] federation.caching = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.789702 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] federation.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.789813 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.789931 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.790056 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.790169 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.790281 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.790392 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.790503 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.790625 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.790738 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.790850 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.790961 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.791071 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.791198 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.791318 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.791432 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.791542 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.791660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.791772 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.791891 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.792005 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.792116 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.792226 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.792337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] identity.caching = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.792448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.792637 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.792753 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.792864 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.792974 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.793091 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.793209 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.793329 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.793441 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.793553 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.793667 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.793779 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.793889 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.794026 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.794140 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.794264 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.794377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.794488 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.794599 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.794718 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.794830 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.794941 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.795051 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.795167 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.795278 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.795397 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.795509 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.795621 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.795731 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.795841 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.795952 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.795952 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.796174 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.796286 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.796397 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.796507 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.796618 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.796744 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.796857 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.796968 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.797078 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.797188 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.797299 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.797417 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.797529 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.797529 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.797737 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.797848 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.797964 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.798140 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.798255 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.798368 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.798485 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.798598 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.798710 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.798830 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.798942 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.799058 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.799171 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.799288 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.799401 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.799523 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.799641 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.799753 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.799864 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.799975 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.800085 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.800204 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.800316 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.800426 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.800537 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.800647 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.800765 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.800879 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.800990 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.801100 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.801210 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.801321 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.801440 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.801552 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.801662 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.801772 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.801884 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.801995 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.802133 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.802246 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.802357 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.802468 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.802578 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.802687 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.802807 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.802921 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.803032 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.803142 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.803253 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.803364 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.803482 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.803597 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.803759 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.803885 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.803998 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.804109 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.user = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.804227 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.804340 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.804451 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.804561 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.804671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.804782 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.804900 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.805013 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.805123 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.805233 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.805345 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.805455 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.805574 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.805688 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.805798 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.805909 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.806094 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.806211 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.806335 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.806449 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.806560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.806671 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.806782 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.806893 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.807012 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.807125 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.807236 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.807347 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.807457 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.807567 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.807685 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.807796 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.807909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.808019 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.808129 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.808241 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.808360 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.808472 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.808582 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.808693 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.808803 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.808916 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.809035 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.809154 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.809268 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.809379 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.809489 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.809599 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.809718 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.809830 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.809941 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.810075 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.810189 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.810302 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.810421 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.810533 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.810644 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.810754 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.810865 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.810975 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.811094 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.811206 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.811317 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.811428 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.811539 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.811649 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.811649 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.811945 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.812058 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.812168 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.812278 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.812422 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.812544 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] resource.caching = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.812657 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.812769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.812873 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] resource.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.812873 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.812873 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.812873 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.812873 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.812873 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.813474 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.813474 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.813684 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.813805 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.813917 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.814046 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.814159 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.814375 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.814491 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] role.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.814687 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.814803 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] role.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.814915 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.815075 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.815190 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.815309 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.815535 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] role.caching = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.815651 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.815782 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] role.driver = sql {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.816011 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.816125 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.816245 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.816486 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.816602 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.816713 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.816966 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.817081 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.817202 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.817432 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.817548 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.817661 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.817901 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.818072 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.818200 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.818382 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.818496 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.818628 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.818744 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.818855 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.818974 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.819087 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.819198 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.819376 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.819490 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.819629 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.819754 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.819866 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.820283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.820521 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.820772 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.821022 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.821211 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.821328 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.821481 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.821596 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.821729 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.821857 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.821971 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.822099 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.822246 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.822360 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.822471 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.822595 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.822708 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.822917 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.823148 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.823440 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.823556 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.823699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.823940 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.824056 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.824167 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.824403 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.824517 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.824659 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.824842 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.824959 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.825151 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.825293 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.825409 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.825614 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.825729 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.825841 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.826094 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] token.caching = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.826217 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] token.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.826330 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.826541 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.826807 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.827049 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.827336 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.827588 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.827830 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.828078 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.828323 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.828578 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.828774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.828964 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.829153 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.829345 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.829520 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.829642 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.829754 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.829873 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.829985 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.830129 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.830244 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.830356 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.830477 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.830597 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.830710 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.830821 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.830940 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.831053 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.connection = **** {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.831166 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.831278 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.831396 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.831508 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.831627 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.831740 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.831851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.831969 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.832081 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.832201 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.832308 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.832423 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.832423 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.832652 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.832765 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.832957 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.833198 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.833431 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.833677 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.833911 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.834193 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.834421 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.834552 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.834658 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.834831 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] database.connection = **** {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.835166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.835166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 31 05:04:43.835299 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.835299 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.835549 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.835793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.836031 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.836267 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.836502 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.836736 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.836978 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.837211 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.837450 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.837684 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.837927 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.838184 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.838424 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.838650 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.838888 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.839119 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 05:04:43.839286 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 31 05:04:43.864500 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:43.864688 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:43.864940 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:43.868148 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:43.868374 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:43.868591 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:43.873568 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:43.876996 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:43.901325 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:43.901458 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:43.902520 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:43.902752 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:43.953543 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:43.953956 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.001653 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.001898 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.002649 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.002877 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.156810 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.156993 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.157121 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.157204 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.157291 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.157415 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.157497 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.157571 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.157646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.157776 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.157856 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.157925 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.180511 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.180691 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.180824 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.180928 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.181037 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.181190 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.181314 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.181441 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.181537 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.181656 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.181736 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.181798 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.181865 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.181950 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.192637 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.193082 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.207178 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.207377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.207632 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.207844 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.208070 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.208301 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.208475 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.210713 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.210916 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.211163 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.211378 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.211585 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.211793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.211984 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 05:04:44.225173 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.225379 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 05:04:44.225503 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.225639 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 05:04:44.225718 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.225907 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 05:04:44.226056 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.226193 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 05:04:44.226272 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.226389 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.226485 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.226675 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.226799 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.226933 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.227180 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.227438 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.236651 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.236775 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.236895 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.236978 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.237176 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.237318 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.237455 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.237534 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.244607 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 05:04:44.244731 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 05:04:44.247479 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 05:04:44.247623 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 05:04:44.247761 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.247842 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.248003 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.248094 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.248309 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.248433 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.248570 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.248651 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.248797 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.248898 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.249093 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.249217 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.249351 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.249430 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.249644 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.249767 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.249891 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.249983 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.250201 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.250326 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.272619 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.272750 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.274048 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 05:04:44.274173 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 05:04:44.274312 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.274416 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.278905 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 05:04:44.279031 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 05:04:44.279236 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.279362 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.279512 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.279594 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.285692 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 05:04:44.285817 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 05:04:44.285951 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.286237 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.290037 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 05:04:44.290158 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 05:04:44.290292 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.290410 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.290719 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.290860 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.290977 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.291065 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.291282 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.291405 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.291559 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.291812 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.292080 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.293263 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.293548 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.293832 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.308919 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.309044 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.309184 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.309336 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.312986 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 05:04:44.313132 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 05:04:44.316167 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.316295 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.316428 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.316508 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.320593 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 05:04:44.320721 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 05:04:44.323662 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 05:04:44.323790 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 05:04:44.326792 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.326938 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.327134 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.327262 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.327433 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.327516 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.327695 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.327819 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.327959 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.328040 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.328231 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.328373 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.343732 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 05:04:44.343858 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 05:04:44.343995 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.344181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.344313 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.344410 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.344477 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.344705 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.351270 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.351396 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.352775 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.352997 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.354297 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.354565 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.354693 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.354883 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.357159 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 05:04:44.357437 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 05:04:44.357566 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.357763 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.357886 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.358062 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.358149 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.358300 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.358437 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.358647 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.369929 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 05:04:44.370242 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.370365 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 05:04:44.370622 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.370747 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.370901 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.370985 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.371168 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.371292 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.371427 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.371547 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.371712 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.371803 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.371996 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.386990 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 05:04:44.387181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 05:04:44.389848 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 05:04:44.390295 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 05:04:44.392738 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.393159 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.394171 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.394618 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.395756 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 05:04:44.396035 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.396157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 05:04:44.396293 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.396392 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.396619 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.402559 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.403237 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.404003 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 05:04:44.404694 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 05:04:44.407039 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.407280 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.407558 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.407680 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.407903 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.407998 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.408128 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.408404 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.415577 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 05:04:44.415861 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.416131 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.416253 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 05:04:44.416544 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.416821 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.422975 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 05:04:44.423251 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.423521 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.423753 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 05:04:44.423892 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.424028 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.424108 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.424276 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.424367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.424569 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.424693 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.424828 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.424920 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.425053 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.425138 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.425355 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.425494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.425578 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.425652 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.425856 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.425981 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.426157 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.426241 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.426404 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.426530 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.426687 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.426951 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.427243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 05:04:44.459655 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55a53ed489c0 pid: 81601 (default app) Aug 31 05:04:44.460639 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55a53ed489c0 pid: 81602 (default app) Aug 31 05:04:44.462914 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08436b77-6cf6-4bc1-b2d7-163a1cb0310d None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:04:44.463197 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08436b77-6cf6-4bc1-b2d7-163a1cb0310d None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:04:44.463465 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08436b77-6cf6-4bc1-b2d7-163a1cb0310d None None] PATH_INFO: `/v3` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:04:44.467742 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1/1] 10.222.0.36 () {50 vars in 791 bytes} [Wed Aug 31 05:04:44 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:04:51.035756 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf09ac-7953-4857-8b81-50a46740d166 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:04:51.036293 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf09ac-7953-4857-8b81-50a46740d166 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:04:51.036403 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf09ac-7953-4857-8b81-50a46740d166 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:04:51.040722 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1/2] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:04:51 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:04:51.047187 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b01d9a-99ba-41c6-b361-4c4b1e4a55d9 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:04:51.047655 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b01d9a-99ba-41c6-b361-4c4b1e4a55d9 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:04:51.047793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b01d9a-99ba-41c6-b361-4c4b1e4a55d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:04:51.127991 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: INFO dbcounter [None req-74b01d9a-99ba-41c6-b361-4c4b1e4a55d9 None None] Registered counter for database keystone Aug 31 05:04:51.149877 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG oslo_db.sqlalchemy.engines [None req-74b01d9a-99ba-41c6-b361-4c4b1e4a55d9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81601) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 31 05:04:51.157757 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writer thread running {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 31 05:04:51.345924 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG passlib.handlers.bcrypt [None req-74b01d9a-99ba-41c6-b361-4c4b1e4a55d9 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81601) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 31 05:04:51.346244 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG passlib.handlers.bcrypt [None req-74b01d9a-99ba-41c6-b361-4c4b1e4a55d9 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81601) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 31 05:04:51.404985 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-74b01d9a-99ba-41c6-b361-4c4b1e4a55d9 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:04:51.406628 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-74b01d9a-99ba-41c6-b361-4c4b1e4a55d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:04:51.562408 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 2/3] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:04:51 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 517 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 31 05:04:52.047641 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaf09ac-7953-4857-8b81-50a46740d166 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:04:52.122796 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: INFO dbcounter [None req-7eaf09ac-7953-4857-8b81-50a46740d166 None None] Registered counter for database keystone Aug 31 05:04:52.147872 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG oslo_db.sqlalchemy.engines [None req-7eaf09ac-7953-4857-8b81-50a46740d166 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81602) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 31 05:04:52.156998 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writer thread running {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 31 05:04:52.287521 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaf09ac-7953-4857-8b81-50a46740d166 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:04:52.292886 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a8b114-0a33-44dd-9daa-38d38c0d0905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:04:52.293520 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8b114-0a33-44dd-9daa-38d38c0d0905 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:04:52.293801 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8b114-0a33-44dd-9daa-38d38c0d0905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:04:52.294108 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8b114-0a33-44dd-9daa-38d38c0d0905 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:04:52.303404 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a8b114-0a33-44dd-9daa-38d38c0d0905 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:04:52.355130 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a6a8b114-0a33-44dd-9daa-38d38c0d0905 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 31 05:04:52.355475 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a8b114-0a33-44dd-9daa-38d38c0d0905 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:04:52.365820 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 2/4] 10.222.0.36 () {58 vars in 1324 bytes} [Wed Aug 31 05:04:52 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 05:04:52.375478 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b01d9a-99ba-41c6-b361-4c4b1e4a55d9 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:04:52.390937 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b01d9a-99ba-41c6-b361-4c4b1e4a55d9 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:04:52.398960 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c555c8af-b6a7-4dcd-b373-5d8970aa9ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:04:52.400618 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c555c8af-b6a7-4dcd-b373-5d8970aa9ac0 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:04:52.400989 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c555c8af-b6a7-4dcd-b373-5d8970aa9ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:04:52.401319 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c555c8af-b6a7-4dcd-b373-5d8970aa9ac0 admin admin] PATH_INFO: `/v3/projects/3e3bdb87df7f4087806a21456cb45a01` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:04:52.404102 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c555c8af-b6a7-4dcd-b373-5d8970aa9ac0 admin admin] RBAC: Authorizing `identity:get_project(project_id=3e3bdb87df7f4087806a21456cb45a01)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:04:52.463559 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c555c8af-b6a7-4dcd-b373-5d8970aa9ac0 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 31 05:04:52.463916 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c555c8af-b6a7-4dcd-b373-5d8970aa9ac0 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:04:52.468518 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 3/5] 10.222.0.36 () {56 vars in 1207 bytes} [Wed Aug 31 05:04:52 2022] GET /identity/v3/projects/3e3bdb87df7f4087806a21456cb45a01 => generated 344 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:04:54.366578 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375151d-72a0-4031-a4e8-875dcb51eb50 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:04:54.367170 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375151d-72a0-4031-a4e8-875dcb51eb50 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:04:54.367170 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375151d-72a0-4031-a4e8-875dcb51eb50 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:04:54.371625 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 3/6] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:04:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:04:54.378422 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1910d-0b4f-4c7e-bc2d-7b44ca882ff4 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:04:54.379113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1910d-0b4f-4c7e-bc2d-7b44ca882ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:04:54.379260 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1910d-0b4f-4c7e-bc2d-7b44ca882ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:04:54.410945 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-dee1910d-0b4f-4c7e-bc2d-7b44ca882ff4 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:04:54.412543 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-dee1910d-0b4f-4c7e-bc2d-7b44ca882ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:04:54.422049 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 4/7] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:04:54 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 31 05:04:54.689548 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b375151d-72a0-4031-a4e8-875dcb51eb50 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:04:54.703392 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b375151d-72a0-4031-a4e8-875dcb51eb50 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:04:54.708777 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6177e677-324a-4803-b536-6a1db4299989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:04:54.709380 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177e677-324a-4803-b536-6a1db4299989 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:04:54.709658 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177e677-324a-4803-b536-6a1db4299989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:04:54.709930 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177e677-324a-4803-b536-6a1db4299989 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:04:54.718536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6177e677-324a-4803-b536-6a1db4299989 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:04:54.720311 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6177e677-324a-4803-b536-6a1db4299989 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:04:54.729781 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 4/8] 10.222.0.36 () {58 vars in 1324 bytes} [Wed Aug 31 05:04:54 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 05:04:54.735386 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee1910d-0b4f-4c7e-bc2d-7b44ca882ff4 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:04:54.745570 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee1910d-0b4f-4c7e-bc2d-7b44ca882ff4 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:04:54.751199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f134754-6d7e-4f42-b850-376c03425636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:04:54.751826 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f134754-6d7e-4f42-b850-376c03425636 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:04:54.752106 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f134754-6d7e-4f42-b850-376c03425636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:04:54.752383 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f134754-6d7e-4f42-b850-376c03425636 admin admin] PATH_INFO: `/v3/users/c96b7a6c0bc940caab4581734c1919ee` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:04:54.753807 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f134754-6d7e-4f42-b850-376c03425636 admin admin] RBAC: Authorizing `identity:get_user(user_id=c96b7a6c0bc940caab4581734c1919ee)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:04:54.755691 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f134754-6d7e-4f42-b850-376c03425636 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:04:54.757235 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 5/9] 10.222.0.36 () {56 vars in 1198 bytes} [Wed Aug 31 05:04:54 2022] GET /identity/v3/users/c96b7a6c0bc940caab4581734c1919ee => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:04:56.702363 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc80f58b-353e-4f83-aaf9-f44310f9ff33 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:04:56.702917 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc80f58b-353e-4f83-aaf9-f44310f9ff33 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:04:56.702917 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc80f58b-353e-4f83-aaf9-f44310f9ff33 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:04:56.704253 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 5/10] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:04:56.709803 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54456d-ee56-4ca2-9ebc-0c2c9c02d6b4 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:04:56.710262 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54456d-ee56-4ca2-9ebc-0c2c9c02d6b4 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:04:56.710371 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54456d-ee56-4ca2-9ebc-0c2c9c02d6b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:04:56.729290 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-3e54456d-ee56-4ca2-9ebc-0c2c9c02d6b4 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:04:56.730521 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3e54456d-ee56-4ca2-9ebc-0c2c9c02d6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:04:56.747539 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9744c4-51d6-4363-94e0-cf12ec756ea8 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:04:56.748059 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9744c4-51d6-4363-94e0-cf12ec756ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:04:56.748347 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9744c4-51d6-4363-94e0-cf12ec756ea8 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:04:56.749725 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 6/11] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 05:04:56.754787 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75675df1-6f42-406b-95fa-1f95115c5830 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:04:56.755068 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75675df1-6f42-406b-95fa-1f95115c5830 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:04:56.755347 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75675df1-6f42-406b-95fa-1f95115c5830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:04:56.798449 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG passlib.handlers.bcrypt [None req-75675df1-6f42-406b-95fa-1f95115c5830 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81602) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 31 05:04:56.798837 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG passlib.handlers.bcrypt [None req-75675df1-6f42-406b-95fa-1f95115c5830 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81602) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 31 05:04:56.811356 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 6/12] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:04:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:04:56.847560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-75675df1-6f42-406b-95fa-1f95115c5830 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:04:56.850199 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-75675df1-6f42-406b-95fa-1f95115c5830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:04:56.951654 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 7/13] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:04:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 05:04:57.076384 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54456d-ee56-4ca2-9ebc-0c2c9c02d6b4 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:04:57.102584 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:04:57.188803 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54456d-ee56-4ca2-9ebc-0c2c9c02d6b4 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:04:57.210108 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75675df1-6f42-406b-95fa-1f95115c5830 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:04:57.224138 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458c8e34-9b7e-4de5-a96f-d5c3d7ef69d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:04:57.224942 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c8e34-9b7e-4de5-a96f-d5c3d7ef69d4 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:04:57.225763 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c8e34-9b7e-4de5-a96f-d5c3d7ef69d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:04:57.226076 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c8e34-9b7e-4de5-a96f-d5c3d7ef69d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:04:57.237944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458c8e34-9b7e-4de5-a96f-d5c3d7ef69d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:04:57.247232 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-458c8e34-9b7e-4de5-a96f-d5c3d7ef69d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:04:57.247232 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:04:57.247232 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:04:57.247232 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:04:57.247232 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:04:57.247232 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:04:57.247232 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:04:57.247232 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:04:57.247232 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:04:57.247232 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:04:57.247232 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:04:57.247232 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:04:57.247232 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:04:57.247232 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:04:57.247232 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:04:57.247232 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:04:57.248144 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:04:57.248144 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:04:57.248144 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:04:57.248144 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:04:57.248144 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:04:57.248144 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:04:57.248144 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:04:57.248144 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:04:57.248144 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:04:57.248144 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:04:57.248144 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:04:57.248144 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:04:57.248144 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:04:57.248144 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:04:57.248144 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:04:57.248144 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:04:57.248971 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:04:57.248971 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:04:57.248971 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:04:57.248971 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:04:57.248971 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:04:57.248971 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:04:57.248971 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:04:57.248971 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:04:57.248971 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:04:57.248971 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:04:57.248971 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:04:57.248971 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:04:57.248971 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 7/14] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 05:04:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:04:57.254274 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458c8e34-9b7e-4de5-a96f-d5c3d7ef69d4 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:04:57.315847 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75675df1-6f42-406b-95fa-1f95115c5830 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:04:57.318167 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:04:57.335060 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:04:57.352938 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714344aa-1190-4f40-a47b-b05d7b68b1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:04:57.353195 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458c8e34-9b7e-4de5-a96f-d5c3d7ef69d4 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:04:57.353818 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714344aa-1190-4f40-a47b-b05d7b68b1df None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:04:57.354140 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714344aa-1190-4f40-a47b-b05d7b68b1df None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:04:57.354424 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714344aa-1190-4f40-a47b-b05d7b68b1df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:04:57.387413 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605fb857-381e-41bc-90cd-1bfd501476b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:04:57.388018 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605fb857-381e-41bc-90cd-1bfd501476b3 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:04:57.388289 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605fb857-381e-41bc-90cd-1bfd501476b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:04:57.388546 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605fb857-381e-41bc-90cd-1bfd501476b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:04:57.389216 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605fb857-381e-41bc-90cd-1bfd501476b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:04:57.389514 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605fb857-381e-41bc-90cd-1bfd501476b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:04:57.390967 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605fb857-381e-41bc-90cd-1bfd501476b3 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:04:57.402989 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 8/15] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:04:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:04:57.408717 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605fb857-381e-41bc-90cd-1bfd501476b3 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:04:57.423403 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714344aa-1190-4f40-a47b-b05d7b68b1df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:04:57.425125 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714344aa-1190-4f40-a47b-b05d7b68b1df None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:04:57.507123 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605fb857-381e-41bc-90cd-1bfd501476b3 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:04:57.521315 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 8/16] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:04:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 05:04:57.527076 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714344aa-1190-4f40-a47b-b05d7b68b1df None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:04:57.529704 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:04:57.541064 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61be7d6f-6c49-49bf-b869-e922013f8dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:04:57.541701 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61be7d6f-6c49-49bf-b869-e922013f8dd8 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:04:57.541983 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61be7d6f-6c49-49bf-b869-e922013f8dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:04:57.542285 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61be7d6f-6c49-49bf-b869-e922013f8dd8 None admin] PATH_INFO: `/v3/users/c96b7a6c0bc940caab4581734c1919ee` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:04:57.543431 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61be7d6f-6c49-49bf-b869-e922013f8dd8 None admin] RBAC: Authorizing `identity:get_user(user_id=c96b7a6c0bc940caab4581734c1919ee)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:04:57.544915 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61be7d6f-6c49-49bf-b869-e922013f8dd8 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:04:57.546550 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 9/17] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:04:57 2022] GET /identity/v3/users/c96b7a6c0bc940caab4581734c1919ee => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:04:57.552035 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61be7d6f-6c49-49bf-b869-e922013f8dd8 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:04:57.567806 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:04:57.626496 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714344aa-1190-4f40-a47b-b05d7b68b1df None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:04:57.656635 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61be7d6f-6c49-49bf-b869-e922013f8dd8 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:04:57.661263 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c33763e-8b58-46d3-9039-9b1040eb15f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:04:57.662150 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c33763e-8b58-46d3-9039-9b1040eb15f2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:04:57.662425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c33763e-8b58-46d3-9039-9b1040eb15f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:04:57.662685 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c33763e-8b58-46d3-9039-9b1040eb15f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:04:57.681091 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c33763e-8b58-46d3-9039-9b1040eb15f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:04:57.682607 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c33763e-8b58-46d3-9039-9b1040eb15f2 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:04:57.689931 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcd3ac1-93aa-40a3-b1da-9f8b739b0ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:04:57.690604 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd3ac1-93aa-40a3-b1da-9f8b739b0ec8 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:04:57.691020 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd3ac1-93aa-40a3-b1da-9f8b739b0ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:04:57.691287 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd3ac1-93aa-40a3-b1da-9f8b739b0ec8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:04:57.692523 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcd3ac1-93aa-40a3-b1da-9f8b739b0ec8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:04:57.693990 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcd3ac1-93aa-40a3-b1da-9f8b739b0ec8 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:04:57.695754 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 10/18] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:04:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:04:57.700559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-8c33763e-8b58-46d3-9039-9b1040eb15f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:04:57.700559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:04:57.700559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:04:57.700559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:04:57.700559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:04:57.700559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:04:57.700559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:04:57.700559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:04:57.700559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:04:57.700559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:04:57.700559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:04:57.700559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:04:57.700559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:04:57.700559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:04:57.700559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:04:57.700559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:04:57.701485 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:04:57.701485 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:04:57.701485 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:04:57.701485 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:04:57.701485 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:04:57.701485 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:04:57.701485 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:04:57.701485 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:04:57.701485 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:04:57.701485 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:04:57.701485 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:04:57.701485 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:04:57.701485 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:04:57.701485 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:04:57.701485 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:04:57.701485 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:04:57.701485 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:04:57.702314 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:04:57.702314 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:04:57.702314 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:04:57.703013 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcd3ac1-93aa-40a3-b1da-9f8b739b0ec8 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:04:57.703090 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 9/19] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:04:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:04:57.708991 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c33763e-8b58-46d3-9039-9b1040eb15f2 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:04:57.766571 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:04:57.786863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:04:57.805940 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcd3ac1-93aa-40a3-b1da-9f8b739b0ec8 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:04:57.814104 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c33763e-8b58-46d3-9039-9b1040eb15f2 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:04:57.841004 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850b74e7-5acd-4665-8970-bf84b00a6e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:04:57.841613 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b74e7-5acd-4665-8970-bf84b00a6e1e None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:04:57.841916 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b74e7-5acd-4665-8970-bf84b00a6e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:04:57.842207 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b74e7-5acd-4665-8970-bf84b00a6e1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:04:57.842931 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850b74e7-5acd-4665-8970-bf84b00a6e1e None admin] RBAC: Adding query filter params (role.id=c3d5589dd2864be9a0f26f5802d874cd, scope.domain.id=default, user.id=c96b7a6c0bc940caab4581734c1919ee) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:04:57.843249 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850b74e7-5acd-4665-8970-bf84b00a6e1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:04:57.844722 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850b74e7-5acd-4665-8970-bf84b00a6e1e None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:04:57.847591 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0332d8-736c-47e0-b832-6c7214933828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:04:57.848193 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0332d8-736c-47e0-b832-6c7214933828 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:04:57.848451 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0332d8-736c-47e0-b832-6c7214933828 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:04:57.848715 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0332d8-736c-47e0-b832-6c7214933828 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:04:57.849348 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0332d8-736c-47e0-b832-6c7214933828 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:04:57.849645 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0332d8-736c-47e0-b832-6c7214933828 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:04:57.851033 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0332d8-736c-47e0-b832-6c7214933828 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:04:57.851482 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 11/20] 10.222.0.36 () {56 vars in 1322 bytes} [Wed Aug 31 05:04:57 2022] GET /identity/v3/role_assignments?user.id=c96b7a6c0bc940caab4581734c1919ee&scope.domain.id=default&role.id=c3d5589dd2864be9a0f26f5802d874cd => generated 233 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:04:57.870385 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 10/21] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:04:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:04:59.864601 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c79d6-9100-468d-9884-e0178046a33a None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:04:59.865128 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c79d6-9100-468d-9884-e0178046a33a None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:04:59.865128 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c79d6-9100-468d-9884-e0178046a33a None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:04:59.866691 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 12/22] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:04:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:04:59.884694 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf5cdd0-fe13-4b8b-8bcf-3c9165f94055 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:04:59.885204 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf5cdd0-fe13-4b8b-8bcf-3c9165f94055 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:04:59.885289 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf5cdd0-fe13-4b8b-8bcf-3c9165f94055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:04:59.911797 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-acf5cdd0-fe13-4b8b-8bcf-3c9165f94055 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:04:59.912929 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-acf5cdd0-fe13-4b8b-8bcf-3c9165f94055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:00.004103 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 11/23] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:04:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:00.012902 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4106fa-b0a3-489d-a512-f958966230e2 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:00.013221 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4106fa-b0a3-489d-a512-f958966230e2 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:00.013462 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4106fa-b0a3-489d-a512-f958966230e2 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:00.014845 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 13/24] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:00.020576 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b5698-9961-4c70-8895-0ab2f5302259 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:00.020866 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b5698-9961-4c70-8895-0ab2f5302259 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:00.021144 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b5698-9961-4c70-8895-0ab2f5302259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:00.054348 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-fb6b5698-9961-4c70-8895-0ab2f5302259 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:00.055534 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fb6b5698-9961-4c70-8895-0ab2f5302259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:00.116142 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:00.146303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 12/25] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 05:05:00.495423 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4106fa-b0a3-489d-a512-f958966230e2 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:00.599035 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd363116-5942-4b82-9a84-285e9b6b67bd None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:00.599422 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd363116-5942-4b82-9a84-285e9b6b67bd None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:00.599779 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd363116-5942-4b82-9a84-285e9b6b67bd None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:00.601391 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 13/26] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:00.609253 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048483c9-7ba0-4633-a4d8-f37ebb1fe9cc None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:00.609253 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048483c9-7ba0-4633-a4d8-f37ebb1fe9cc None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:00.609253 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048483c9-7ba0-4633-a4d8-f37ebb1fe9cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:00.643719 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-048483c9-7ba0-4633-a4d8-f37ebb1fe9cc None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:00.645015 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-048483c9-7ba0-4633-a4d8-f37ebb1fe9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:00.645531 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4106fa-b0a3-489d-a512-f958966230e2 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:00.693914 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6415cb1-8c52-4bc9-b81b-fe4ef20966df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:00.694669 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6415cb1-8c52-4bc9-b81b-fe4ef20966df None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:00.695101 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6415cb1-8c52-4bc9-b81b-fe4ef20966df None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:00.695496 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6415cb1-8c52-4bc9-b81b-fe4ef20966df None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:00.706177 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6415cb1-8c52-4bc9-b81b-fe4ef20966df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:00.711592 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-f6415cb1-8c52-4bc9-b81b-fe4ef20966df None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:05:00.711592 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:00.711592 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:00.711592 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:00.711592 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:00.711592 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:00.711592 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:00.711592 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:00.711592 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:00.711592 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:00.711592 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:00.711592 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:00.711592 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:00.711592 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:00.711592 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:00.711592 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:00.712630 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:00.712630 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:00.712630 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:00.712630 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:00.712630 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:00.712630 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:00.712630 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:00.712630 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:00.712630 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:00.712630 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:00.712630 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:00.712630 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:00.712630 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:00.712630 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:00.712630 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:00.712630 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:00.713584 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:00.713584 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:00.713584 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:00.713584 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:00.713584 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:00.713584 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:00.713584 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:00.713584 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:00.713584 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:00.713584 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:00.713584 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:05:00.713584 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:05:00.715812 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 14/27] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 05:05:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:05:00.716731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6415cb1-8c52-4bc9-b81b-fe4ef20966df None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:00.730733 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:00.730902 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 14/28] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 05:05:00.732816 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51992db1-65ef-40f1-999a-6ca316bf3196 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:00.733262 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51992db1-65ef-40f1-999a-6ca316bf3196 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:00.733813 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51992db1-65ef-40f1-999a-6ca316bf3196 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:00.735255 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 15/29] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:00.740821 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51992db1-65ef-40f1-999a-6ca316bf3196 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:00.825258 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6415cb1-8c52-4bc9-b81b-fe4ef20966df None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:00.851246 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51992db1-65ef-40f1-999a-6ca316bf3196 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:00.859400 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:00.860109 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eb85b2-7025-446b-8dba-0a41ed760c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:00.860925 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb85b2-7025-446b-8dba-0a41ed760c9c None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:00.861336 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb85b2-7025-446b-8dba-0a41ed760c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:00.861739 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb85b2-7025-446b-8dba-0a41ed760c9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:00.873060 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eb85b2-7025-446b-8dba-0a41ed760c9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:00.875288 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eb85b2-7025-446b-8dba-0a41ed760c9c None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:00.891635 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-62eb85b2-7025-446b-8dba-0a41ed760c9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:05:00.891635 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:00.891635 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:00.891635 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:00.891635 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:00.891635 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:00.891635 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:00.891635 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:00.891635 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:00.891635 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:00.891635 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:00.891635 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:00.891635 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:05:00.891635 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:05:00.891635 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:05:00.891635 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:05:00.892515 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:00.892515 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:00.892515 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:00.892515 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:00.892515 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:00.892515 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:00.892515 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:00.892515 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:00.892515 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:00.892515 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:00.892515 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:00.892515 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:00.892515 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:00.892515 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:00.892515 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:00.892515 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:00.892515 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:05:00.893324 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:05:00.893324 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:05:00.893324 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:05:00.893477 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9ab4f-ca6d-4680-b631-071314d0cbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:00.893477 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9ab4f-ca6d-4680-b631-071314d0cbdf None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:00.893477 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9ab4f-ca6d-4680-b631-071314d0cbdf None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:00.893477 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9ab4f-ca6d-4680-b631-071314d0cbdf None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:00.894842 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a9ab4f-ca6d-4680-b631-071314d0cbdf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:00.894842 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a9ab4f-ca6d-4680-b631-071314d0cbdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:00.895992 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 15/30] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:05:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:05:00.896504 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a9ab4f-ca6d-4680-b631-071314d0cbdf None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:00.897655 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca19c5-69aa-4deb-8827-f2408292672e None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:00.898157 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca19c5-69aa-4deb-8827-f2408292672e None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:00.898616 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca19c5-69aa-4deb-8827-f2408292672e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:00.916268 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 16/31] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:05:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:00.921084 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9ab4f-ca6d-4680-b631-071314d0cbdf None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:00.922830 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-adca19c5-69aa-4deb-8827-f2408292672e None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:00.924256 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-adca19c5-69aa-4deb-8827-f2408292672e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:00.975430 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:00.993204 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 16/32] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:00.994374 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adca19c5-69aa-4deb-8827-f2408292672e None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:01.045091 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9ab4f-ca6d-4680-b631-071314d0cbdf None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:01.094094 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adca19c5-69aa-4deb-8827-f2408292672e None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:01.098361 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd882e-8b8d-49c6-a841-e1db1366cacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:01.098981 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd882e-8b8d-49c6-a841-e1db1366cacc None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:01.099256 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd882e-8b8d-49c6-a841-e1db1366cacc None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:01.099522 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd882e-8b8d-49c6-a841-e1db1366cacc None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:01.100169 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bd882e-8b8d-49c6-a841-e1db1366cacc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:01.100464 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bd882e-8b8d-49c6-a841-e1db1366cacc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:01.101850 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bd882e-8b8d-49c6-a841-e1db1366cacc None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:01.120468 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 17/33] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:05:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:01.121383 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd882e-8b8d-49c6-a841-e1db1366cacc None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:01.133361 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd21127-138a-4986-ad8a-cdfd1cac1235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:01.134321 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd21127-138a-4986-ad8a-cdfd1cac1235 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:01.134813 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd21127-138a-4986-ad8a-cdfd1cac1235 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:01.135288 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd21127-138a-4986-ad8a-cdfd1cac1235 None admin] PATH_INFO: `/v3/users/c96b7a6c0bc940caab4581734c1919ee` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:01.136634 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd21127-138a-4986-ad8a-cdfd1cac1235 None admin] RBAC: Authorizing `identity:get_user(user_id=c96b7a6c0bc940caab4581734c1919ee)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:01.138331 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd21127-138a-4986-ad8a-cdfd1cac1235 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:01.140488 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 17/34] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:05:00 2022] GET /identity/v3/users/c96b7a6c0bc940caab4581734c1919ee => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:01.141441 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd21127-138a-4986-ad8a-cdfd1cac1235 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:01.192119 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:01.203126 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:01.246755 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd21127-138a-4986-ad8a-cdfd1cac1235 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:01.281857 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f574b59c-73e1-4edf-95a7-5d3122e9b0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:01.282889 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574b59c-73e1-4edf-95a7-5d3122e9b0cd None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:01.283359 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574b59c-73e1-4edf-95a7-5d3122e9b0cd None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:01.283816 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574b59c-73e1-4edf-95a7-5d3122e9b0cd None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:01.284658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f574b59c-73e1-4edf-95a7-5d3122e9b0cd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:01.286337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f574b59c-73e1-4edf-95a7-5d3122e9b0cd None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:01.348344 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd882e-8b8d-49c6-a841-e1db1366cacc None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:01.407554 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac62376-d04a-44d9-9b5c-8c6fb86fe66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:01.408520 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac62376-d04a-44d9-9b5c-8c6fb86fe66e None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:01.409012 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac62376-d04a-44d9-9b5c-8c6fb86fe66e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:01.409469 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac62376-d04a-44d9-9b5c-8c6fb86fe66e None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:01.410392 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac62376-d04a-44d9-9b5c-8c6fb86fe66e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:01.411952 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac62376-d04a-44d9-9b5c-8c6fb86fe66e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:01.468166 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 18/35] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Aug 31 05:05:01 2022] POST /identity/v3/projects => generated 301 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:05:01.469043 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f574b59c-73e1-4edf-95a7-5d3122e9b0cd None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:01.559149 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 18/36] 10.222.0.36 () {60 vars in 1131 bytes} [Wed Aug 31 05:05:01 2022] POST /identity/v3/roles => generated 219 bytes in 438 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:05:01.559961 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac62376-d04a-44d9-9b5c-8c6fb86fe66e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:01.586083 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f574b59c-73e1-4edf-95a7-5d3122e9b0cd None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:01.595027 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:01.653089 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aba440-8ba1-40b3-a5ad-565fef686c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:01.653089 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aba440-8ba1-40b3-a5ad-565fef686c2a None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:01.653089 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aba440-8ba1-40b3-a5ad-565fef686c2a None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:01.653089 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aba440-8ba1-40b3-a5ad-565fef686c2a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:01.656007 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aba440-8ba1-40b3-a5ad-565fef686c2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:01.657745 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aba440-8ba1-40b3-a5ad-565fef686c2a None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:01.659818 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 19/37] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:05:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:01.661200 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12a151-0f10-40e6-81a3-cc6cf9a04a53 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:01.665025 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12a151-0f10-40e6-81a3-cc6cf9a04a53 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:01.665347 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12a151-0f10-40e6-81a3-cc6cf9a04a53 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:01.667186 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 20/38] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:01.670797 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a12a151-0f10-40e6-81a3-cc6cf9a04a53 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:01.693955 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:01.723837 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac62376-d04a-44d9-9b5c-8c6fb86fe66e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:01.767056 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f406173-aa03-4dd6-af01-1b212723dc4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:01.767696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f406173-aa03-4dd6-af01-1b212723dc4c None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:01.768001 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f406173-aa03-4dd6-af01-1b212723dc4c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:01.768377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f406173-aa03-4dd6-af01-1b212723dc4c None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:01.769063 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f406173-aa03-4dd6-af01-1b212723dc4c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:01.770787 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f406173-aa03-4dd6-af01-1b212723dc4c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:01.791482 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 19/39] 10.222.0.36 () {60 vars in 1131 bytes} [Wed Aug 31 05:05:01 2022] POST /identity/v3/roles => generated 225 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:05:01.792691 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6c89c-5724-4629-8a53-bc2d04d79fc8 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:01.792982 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6c89c-5724-4629-8a53-bc2d04d79fc8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:01.793251 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6c89c-5724-4629-8a53-bc2d04d79fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:01.819403 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-aed6c89c-5724-4629-8a53-bc2d04d79fc8 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:01.821292 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-aed6c89c-5724-4629-8a53-bc2d04d79fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:01.844945 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a12a151-0f10-40e6-81a3-cc6cf9a04a53 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:01.892509 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38e4fb2-b4b8-4d60-b2d5-2356da810945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:01.893907 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e4fb2-b4b8-4d60-b2d5-2356da810945 None admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:01.894142 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e4fb2-b4b8-4d60-b2d5-2356da810945 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:01.894423 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e4fb2-b4b8-4d60-b2d5-2356da810945 None admin] PATH_INFO: `/v3/domains/default/users/c96b7a6c0bc940caab4581734c1919ee/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:01.897105 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38e4fb2-b4b8-4d60-b2d5-2356da810945 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=c96b7a6c0bc940caab4581734c1919ee, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:01.899166 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38e4fb2-b4b8-4d60-b2d5-2356da810945 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:01.913241 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 20/40] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:01.920593 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 21/41] 10.222.0.36 () {58 vars in 1361 bytes} [Wed Aug 31 05:05:01 2022] PUT /identity/v3/domains/default/users/c96b7a6c0bc940caab4581734c1919ee/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:05:01.974096 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbad7c6-7e35-4c0c-be4b-c6fa86c6e318 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:01.974417 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbad7c6-7e35-4c0c-be4b-c6fa86c6e318 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:01.974659 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbad7c6-7e35-4c0c-be4b-c6fa86c6e318 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:01.976200 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 21/42] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:01.981754 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8db60c-b06c-4b89-8ca0-c54dcdbdedb9 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:01.982069 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8db60c-b06c-4b89-8ca0-c54dcdbdedb9 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:01.982347 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8db60c-b06c-4b89-8ca0-c54dcdbdedb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:02.004035 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-dd8db60c-b06c-4b89-8ca0-c54dcdbdedb9 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:02.005619 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-dd8db60c-b06c-4b89-8ca0-c54dcdbdedb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:02.038205 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:02.193915 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbad7c6-7e35-4c0c-be4b-c6fa86c6e318 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:02.196093 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 22/43] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 05:05:02.277585 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:02.304030 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbad7c6-7e35-4c0c-be4b-c6fa86c6e318 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:02.338751 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea5b0a9-fcbd-4f32-891b-0d2248e283f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:02.339365 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea5b0a9-fcbd-4f32-891b-0d2248e283f2 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:02.339649 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea5b0a9-fcbd-4f32-891b-0d2248e283f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:02.339923 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea5b0a9-fcbd-4f32-891b-0d2248e283f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:02.340608 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea5b0a9-fcbd-4f32-891b-0d2248e283f2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:02.342013 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea5b0a9-fcbd-4f32-891b-0d2248e283f2 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:02.357216 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 22/44] 10.222.0.36 () {60 vars in 1131 bytes} [Wed Aug 31 05:05:02 2022] POST /identity/v3/roles => generated 223 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:05:02.459936 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8db60c-b06c-4b89-8ca0-c54dcdbdedb9 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:02.568894 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8db60c-b06c-4b89-8ca0-c54dcdbdedb9 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:02.602621 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f603b-5d0e-4bd5-b3ac-33947d2d9326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:02.603255 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f603b-5d0e-4bd5-b3ac-33947d2d9326 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:02.603533 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f603b-5d0e-4bd5-b3ac-33947d2d9326 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:02.603803 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f603b-5d0e-4bd5-b3ac-33947d2d9326 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:02.604914 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f603b-5d0e-4bd5-b3ac-33947d2d9326 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:02.606400 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f603b-5d0e-4bd5-b3ac-33947d2d9326 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:02.608141 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 23/45] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:05:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:02.613680 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea5b0a9-fcbd-4f32-891b-0d2248e283f2 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:02.718331 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea5b0a9-fcbd-4f32-891b-0d2248e283f2 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:02.752381 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963c5beb-21ef-4b9b-a300-897dc82ef37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:02.752992 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c5beb-21ef-4b9b-a300-897dc82ef37b None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:02.753274 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c5beb-21ef-4b9b-a300-897dc82ef37b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:02.753545 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c5beb-21ef-4b9b-a300-897dc82ef37b None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:02.754477 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963c5beb-21ef-4b9b-a300-897dc82ef37b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:02.755951 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963c5beb-21ef-4b9b-a300-897dc82ef37b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:02.766119 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:02.785332 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 23/46] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Aug 31 05:05:02 2022] POST /identity/v3/projects => generated 312 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:05:03.826343 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e7e41-e45d-4470-951a-7e7ac3ffe841 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:03.826897 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e7e41-e45d-4470-951a-7e7ac3ffe841 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:03.826897 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e7e41-e45d-4470-951a-7e7ac3ffe841 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:03.828306 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 24/47] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:03.833687 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1801d-6cf8-4328-966e-271700f349c8 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:03.834209 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1801d-6cf8-4328-966e-271700f349c8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:03.834292 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1801d-6cf8-4328-966e-271700f349c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:03.853423 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-fae1801d-6cf8-4328-966e-271700f349c8 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:03.854642 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fae1801d-6cf8-4328-966e-271700f349c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:03.926226 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 24/48] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:04.191591 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e7e41-e45d-4470-951a-7e7ac3ffe841 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:04.224568 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:04.298528 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e7e41-e45d-4470-951a-7e7ac3ffe841 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:04.333832 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cfe73b-938d-4f8b-a980-89e01bf3594a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:04.334488 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfe73b-938d-4f8b-a980-89e01bf3594a None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:04.334772 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfe73b-938d-4f8b-a980-89e01bf3594a None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:04.335050 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfe73b-938d-4f8b-a980-89e01bf3594a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:04.345312 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cfe73b-938d-4f8b-a980-89e01bf3594a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:04.350365 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-d3cfe73b-938d-4f8b-a980-89e01bf3594a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:05:04.350365 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:04.350365 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:04.350365 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:04.350365 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:04.350365 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:04.350365 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:04.350365 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:04.350365 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:04.350365 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:04.350365 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:04.350365 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:04.350365 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:04.350365 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:04.350365 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:04.350365 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:04.351181 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:04.351181 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:04.351181 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:04.351181 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:04.351181 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:04.351181 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:04.351181 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:04.351181 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:04.351181 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:04.351181 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:04.351181 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:04.351181 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:04.351181 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:04.351181 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:04.351181 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:04.351181 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:04.351936 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:04.351936 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:04.351936 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:04.351936 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:04.351936 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:04.351936 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:04.351936 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:04.351936 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:04.351936 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:04.351936 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:04.351936 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:05:04.351936 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:05:04.351936 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 25/49] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 05:05:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:05:04.357503 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1801d-6cf8-4328-966e-271700f349c8 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:04.460795 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1801d-6cf8-4328-966e-271700f349c8 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:04.480770 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:04.496935 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77595a44-d4c5-4adc-bc1b-6f245d471e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:04.497553 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77595a44-d4c5-4adc-bc1b-6f245d471e7b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:04.497834 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77595a44-d4c5-4adc-bc1b-6f245d471e7b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:04.498140 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77595a44-d4c5-4adc-bc1b-6f245d471e7b None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:04.498818 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77595a44-d4c5-4adc-bc1b-6f245d471e7b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:04.499124 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77595a44-d4c5-4adc-bc1b-6f245d471e7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:04.500522 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77595a44-d4c5-4adc-bc1b-6f245d471e7b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:04.508847 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 25/50] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:05:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:04.515381 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cfe73b-938d-4f8b-a980-89e01bf3594a None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:04.610812 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:04.621164 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cfe73b-938d-4f8b-a980-89e01bf3594a None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:04.662115 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca9a57b-14e1-43eb-aa02-dd9914477334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:04.662750 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca9a57b-14e1-43eb-aa02-dd9914477334 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:04.663030 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca9a57b-14e1-43eb-aa02-dd9914477334 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:04.663303 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca9a57b-14e1-43eb-aa02-dd9914477334 None admin] PATH_INFO: `/v3/users/c96b7a6c0bc940caab4581734c1919ee` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:04.664466 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca9a57b-14e1-43eb-aa02-dd9914477334 None admin] RBAC: Authorizing `identity:get_user(user_id=c96b7a6c0bc940caab4581734c1919ee)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:04.665950 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca9a57b-14e1-43eb-aa02-dd9914477334 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:04.667576 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 26/51] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:05:04 2022] GET /identity/v3/users/c96b7a6c0bc940caab4581734c1919ee => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:04.673606 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77595a44-d4c5-4adc-bc1b-6f245d471e7b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:04.676820 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd57d14b-ba39-4d8f-bca6-a980abecbd34 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:04.677170 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd57d14b-ba39-4d8f-bca6-a980abecbd34 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:04.677441 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd57d14b-ba39-4d8f-bca6-a980abecbd34 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:04.678765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 27/52] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:04.683670 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109c200-a5f5-4391-a833-67e3961e3517 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:04.683953 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109c200-a5f5-4391-a833-67e3961e3517 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:04.684223 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109c200-a5f5-4391-a833-67e3961e3517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:04.702603 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-1109c200-a5f5-4391-a833-67e3961e3517 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:04.703812 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-1109c200-a5f5-4391-a833-67e3961e3517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:04.769294 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 28/53] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:04.778600 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77595a44-d4c5-4adc-bc1b-6f245d471e7b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:04.813259 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb68ae94-0008-4532-9388-d2557ed01599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:04.813900 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68ae94-0008-4532-9388-d2557ed01599 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:04.814218 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68ae94-0008-4532-9388-d2557ed01599 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:04.814492 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68ae94-0008-4532-9388-d2557ed01599 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:04.815606 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb68ae94-0008-4532-9388-d2557ed01599 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:04.817094 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb68ae94-0008-4532-9388-d2557ed01599 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:04.818756 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 26/54] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:05:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:04.824790 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1109c200-a5f5-4391-a833-67e3961e3517 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:04.923920 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:04.934913 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1109c200-a5f5-4391-a833-67e3961e3517 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:04.970482 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29332df-7114-4279-b9c4-6ab0801eaefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:04.971087 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29332df-7114-4279-b9c4-6ab0801eaefe None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:04.971365 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29332df-7114-4279-b9c4-6ab0801eaefe None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:04.971638 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29332df-7114-4279-b9c4-6ab0801eaefe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:04.972366 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29332df-7114-4279-b9c4-6ab0801eaefe None admin] RBAC: Adding query filter params (role.id=c3d5589dd2864be9a0f26f5802d874cd, scope.domain.id=default, user.id=c96b7a6c0bc940caab4581734c1919ee) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:04.972673 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29332df-7114-4279-b9c4-6ab0801eaefe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:04.974166 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29332df-7114-4279-b9c4-6ab0801eaefe None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:04.980013 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 29/55] 10.222.0.36 () {56 vars in 1322 bytes} [Wed Aug 31 05:05:04 2022] GET /identity/v3/role_assignments?user.id=c96b7a6c0bc940caab4581734c1919ee&scope.domain.id=default&role.id=c3d5589dd2864be9a0f26f5802d874cd => generated 530 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:05.044129 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb68ae94-0008-4532-9388-d2557ed01599 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:05.207691 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:05.258387 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb68ae94-0008-4532-9388-d2557ed01599 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:05.293600 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0a6b75-2316-4cde-a3c4-50b69106830b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:05.294322 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a6b75-2316-4cde-a3c4-50b69106830b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:05.294541 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a6b75-2316-4cde-a3c4-50b69106830b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:05.294815 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a6b75-2316-4cde-a3c4-50b69106830b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:05.296081 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0a6b75-2316-4cde-a3c4-50b69106830b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:05.297594 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0a6b75-2316-4cde-a3c4-50b69106830b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:05.299218 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 27/56] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:05:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:05.305263 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29332df-7114-4279-b9c4-6ab0801eaefe None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:05.408887 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29332df-7114-4279-b9c4-6ab0801eaefe None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:05.444539 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f930ab57-41a5-46f3-ba2d-c958f141e1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:05.445199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930ab57-41a5-46f3-ba2d-c958f141e1d0 None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:05.445513 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930ab57-41a5-46f3-ba2d-c958f141e1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:05.445789 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930ab57-41a5-46f3-ba2d-c958f141e1d0 None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:05.446622 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f930ab57-41a5-46f3-ba2d-c958f141e1d0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:05.448118 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f930ab57-41a5-46f3-ba2d-c958f141e1d0 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:05.479740 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 30/57] 10.222.0.36 () {60 vars in 1140 bytes} [Wed Aug 31 05:05:05 2022] POST /identity/v3/projects => generated 298 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:05:07.367644 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c57ec-e3f2-46cb-bce6-e417eaa16453 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:07.368867 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c57ec-e3f2-46cb-bce6-e417eaa16453 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:07.368867 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c57ec-e3f2-46cb-bce6-e417eaa16453 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:07.371051 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 28/58] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:07.377242 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f81f4-e74a-40b4-8df7-b5abdff5d26b None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:07.377668 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f81f4-e74a-40b4-8df7-b5abdff5d26b None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:07.377772 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f81f4-e74a-40b4-8df7-b5abdff5d26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:07.396424 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:07.402411 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-d26f81f4-e74a-40b4-8df7-b5abdff5d26b None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:07.404437 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-d26f81f4-e74a-40b4-8df7-b5abdff5d26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:07.489801 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 31/59] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:07.753885 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c57ec-e3f2-46cb-bce6-e417eaa16453 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:07.860510 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c57ec-e3f2-46cb-bce6-e417eaa16453 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:07.878035 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:07.895849 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6b46cb-0b75-4ccf-a173-ef37eae5fd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:07.896377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b46cb-0b75-4ccf-a173-ef37eae5fd81 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:07.896649 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b46cb-0b75-4ccf-a173-ef37eae5fd81 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:07.896912 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b46cb-0b75-4ccf-a173-ef37eae5fd81 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:07.898068 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6b46cb-0b75-4ccf-a173-ef37eae5fd81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:07.899568 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6b46cb-0b75-4ccf-a173-ef37eae5fd81 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:07.901097 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 29/60] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:05:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:07.907132 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f81f4-e74a-40b4-8df7-b5abdff5d26b None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:08.020204 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f81f4-e74a-40b4-8df7-b5abdff5d26b None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:08.054958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fe672d-30f2-4e2d-b5eb-c8e365dbf1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:08.055550 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe672d-30f2-4e2d-b5eb-c8e365dbf1b1 None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:08.055832 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe672d-30f2-4e2d-b5eb-c8e365dbf1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:08.056099 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe672d-30f2-4e2d-b5eb-c8e365dbf1b1 None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:08.056760 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fe672d-30f2-4e2d-b5eb-c8e365dbf1b1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:08.058256 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fe672d-30f2-4e2d-b5eb-c8e365dbf1b1 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:08.070174 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:08.085150 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 32/61] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Aug 31 05:05:07 2022] POST /identity/v3/users => generated 275 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:05:10.946418 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19df06-c9fa-483d-b936-774790bba218 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:10.947396 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19df06-c9fa-483d-b936-774790bba218 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:10.947873 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19df06-c9fa-483d-b936-774790bba218 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:10.949484 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 30/62] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:10.954939 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a425962-cb5c-4dbc-aa12-febc3696d3f9 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:10.955728 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a425962-cb5c-4dbc-aa12-febc3696d3f9 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:10.956200 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a425962-cb5c-4dbc-aa12-febc3696d3f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:10.978397 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-7a425962-cb5c-4dbc-aa12-febc3696d3f9 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:10.979896 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-7a425962-cb5c-4dbc-aa12-febc3696d3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:11.069772 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 33/63] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:11.205660 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff168797-725b-4d39-8e51-21380c93d59e None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:11.205952 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff168797-725b-4d39-8e51-21380c93d59e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:11.206170 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff168797-725b-4d39-8e51-21380c93d59e None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:11.207528 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 31/64] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:11.213025 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a1727-e4d9-4821-a826-c9fe7ad334d6 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:11.213311 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a1727-e4d9-4821-a826-c9fe7ad334d6 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:11.213632 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a1727-e4d9-4821-a826-c9fe7ad334d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:11.240422 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-e06a1727-e4d9-4821-a826-c9fe7ad334d6 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:11.241641 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-e06a1727-e4d9-4821-a826-c9fe7ad334d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:11.281942 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f86c2-877c-46f5-927f-c30c7f01e370 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:11.282596 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f86c2-877c-46f5-927f-c30c7f01e370 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:11.283063 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f86c2-877c-46f5-927f-c30c7f01e370 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:11.284744 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 32/65] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:11.289971 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc8402-223d-42bc-8f28-303296a27abc None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:11.290529 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc8402-223d-42bc-8f28-303296a27abc None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:11.290991 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc8402-223d-42bc-8f28-303296a27abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:11.323443 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d3bc8402-223d-42bc-8f28-303296a27abc None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:11.325006 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d3bc8402-223d-42bc-8f28-303296a27abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:11.340206 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 34/66] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:11.372714 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06a1727-e4d9-4821-a826-c9fe7ad334d6 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:11.451865 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:11.454787 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 33/67] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:11.517974 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06a1727-e4d9-4821-a826-c9fe7ad334d6 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:11.552311 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a93c6-a64c-4670-917e-28104871817b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:11.553207 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a93c6-a64c-4670-917e-28104871817b None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:11.553616 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a93c6-a64c-4670-917e-28104871817b None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:11.554034 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a93c6-a64c-4670-917e-28104871817b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:11.564585 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a93c6-a64c-4670-917e-28104871817b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:11.569532 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-b82a93c6-a64c-4670-917e-28104871817b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 05:05:11.569532 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:11.569532 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:11.569532 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:11.569532 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:11.569532 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:11.569532 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:11.569532 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:11.569532 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:11.569532 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:11.569532 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:11.569532 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:11.569532 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:11.569532 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:11.569532 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:11.569532 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:11.570565 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:11.570565 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:11.570565 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:11.570565 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:11.570565 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:11.570565 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:11.570565 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:11.570565 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:11.570565 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:11.570565 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:11.570565 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:11.570565 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:11.570565 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:11.570565 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:11.570565 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:11.570565 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:11.571517 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:11.571517 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:11.571517 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:11.571517 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:11.571517 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:11.571517 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:11.571517 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:11.571517 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:11.571517 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:11.571517 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:11.571517 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 05:05:11.571517 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:05:11.573477 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 35/68] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:05:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:05:11.579741 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc8402-223d-42bc-8f28-303296a27abc None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:11.643783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d255ed8-1b70-4e1c-af92-3267fe57acd7 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:11.644330 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d255ed8-1b70-4e1c-af92-3267fe57acd7 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:11.644821 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d255ed8-1b70-4e1c-af92-3267fe57acd7 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:11.646482 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 36/69] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:11 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:11.663466 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046f581-9c86-437c-8234-7d9331a218c4 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:11.664081 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046f581-9c86-437c-8234-7d9331a218c4 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:11.664551 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046f581-9c86-437c-8234-7d9331a218c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:11.690999 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc8402-223d-42bc-8f28-303296a27abc None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:11.701356 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-5046f581-9c86-437c-8234-7d9331a218c4 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:11.703044 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-5046f581-9c86-437c-8234-7d9331a218c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:11.710419 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:11.729218 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a18f24-80ba-4022-aaaf-28d88e1ecf95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:11.730362 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a18f24-80ba-4022-aaaf-28d88e1ecf95 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:11.730834 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a18f24-80ba-4022-aaaf-28d88e1ecf95 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:11.731314 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a18f24-80ba-4022-aaaf-28d88e1ecf95 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:11.732174 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a18f24-80ba-4022-aaaf-28d88e1ecf95 None admin] RBAC: Adding query filter params (name=member) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:11.732682 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a18f24-80ba-4022-aaaf-28d88e1ecf95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:11.734528 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a18f24-80ba-4022-aaaf-28d88e1ecf95 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:11.748286 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 34/70] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 05:05:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:11.749559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a18f24-80ba-4022-aaaf-28d88e1ecf95 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:11.803209 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 37/71] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:11.807218 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5046f581-9c86-437c-8234-7d9331a218c4 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:11.855416 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a18f24-80ba-4022-aaaf-28d88e1ecf95 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:11.865992 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:11.894738 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac78f88-3658-459e-81be-7748983fa643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:11.895723 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac78f88-3658-459e-81be-7748983fa643 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:11.896195 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac78f88-3658-459e-81be-7748983fa643 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:11.896622 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac78f88-3658-459e-81be-7748983fa643 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:11.909927 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac78f88-3658-459e-81be-7748983fa643 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:11.916335 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-bac78f88-3658-459e-81be-7748983fa643 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:05:11.916335 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:11.916335 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:11.916335 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:11.916335 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:11.916335 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:11.916335 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:11.916335 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:11.916335 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:11.916335 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:11.916335 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:11.916335 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:11.916335 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:11.916335 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:11.916335 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:11.916335 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:11.917191 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:11.917191 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:11.917191 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:11.917191 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:11.917191 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:11.917191 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:11.917191 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:11.917191 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:11.917191 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:11.917191 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:11.917191 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:11.917191 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:11.917191 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:11.917191 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:11.917191 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:11.917191 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:11.917963 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:11.917963 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:11.917963 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:11.917963 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:11.917963 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:11.917963 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:11.917963 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:11.917963 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:11.917963 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:11.917963 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:11.917963 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:05:11.917963 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:05:11.920198 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 35/72] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 05:05:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:05:11.921388 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac78f88-3658-459e-81be-7748983fa643 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:11.940155 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5046f581-9c86-437c-8234-7d9331a218c4 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:11.963909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:11.987292 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40ba16d-6c1d-4160-a54d-314e5b94ee3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:11.987930 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ba16d-6c1d-4160-a54d-314e5b94ee3d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:11.988326 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ba16d-6c1d-4160-a54d-314e5b94ee3d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:11.988715 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ba16d-6c1d-4160-a54d-314e5b94ee3d None admin] PATH_INFO: `/v3/users/bbcee328f8db4f5cb50f75830eb1cb5a` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:12.009109 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40ba16d-6c1d-4160-a54d-314e5b94ee3d None admin] RBAC: Authorizing `identity:get_user(user_id=bbcee328f8db4f5cb50f75830eb1cb5a)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:12.010969 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40ba16d-6c1d-4160-a54d-314e5b94ee3d None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:12.012785 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 38/73] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:05:11 2022] GET /identity/v3/users/bbcee328f8db4f5cb50f75830eb1cb5a => generated 275 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:12.014087 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40ba16d-6c1d-4160-a54d-314e5b94ee3d None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:12.031633 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac78f88-3658-459e-81be-7748983fa643 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:12.065820 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7790f20-88df-46bc-8085-e71a534430ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:12.066765 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7790f20-88df-46bc-8085-e71a534430ff None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:12.067277 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7790f20-88df-46bc-8085-e71a534430ff None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:12.067780 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7790f20-88df-46bc-8085-e71a534430ff None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:12.079342 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7790f20-88df-46bc-8085-e71a534430ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:12.084578 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-d7790f20-88df-46bc-8085-e71a534430ff None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 05:05:12.084578 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:12.084578 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:12.084578 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:12.084578 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:12.084578 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:12.084578 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:12.084578 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:12.084578 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:12.084578 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:12.084578 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:12.084578 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:12.084578 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:12.084578 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:12.084578 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:12.084578 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:12.085579 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:12.085579 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:12.085579 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:12.085579 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:12.085579 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:12.085579 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:12.085579 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:12.085579 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:12.085579 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:12.085579 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:12.085579 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:12.085579 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:12.085579 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:12.085579 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:12.085579 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:12.085579 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:12.086381 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:12.086381 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:12.086381 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:12.086381 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:12.086381 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:12.086381 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:12.086381 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:12.086381 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:12.086381 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:12.086381 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:12.086381 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 05:05:12.086381 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:05:12.086381 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 36/74] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 05:05:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:05:12.087054 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6036b7c-631e-4950-92fc-36ad641d2d62 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:12.087221 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6036b7c-631e-4950-92fc-36ad641d2d62 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:12.087491 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6036b7c-631e-4950-92fc-36ad641d2d62 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:12.088832 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 37/75] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:12.089460 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6036b7c-631e-4950-92fc-36ad641d2d62 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:12.123054 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:12.127040 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40ba16d-6c1d-4160-a54d-314e5b94ee3d None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:12.162355 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba37a0-fbe2-4d4e-b627-2c345581be0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:12.162989 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba37a0-fbe2-4d4e-b627-2c345581be0a None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:12.163249 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba37a0-fbe2-4d4e-b627-2c345581be0a None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:12.163513 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba37a0-fbe2-4d4e-b627-2c345581be0a None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:12.164174 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aba37a0-fbe2-4d4e-b627-2c345581be0a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:12.164469 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aba37a0-fbe2-4d4e-b627-2c345581be0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:12.165864 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aba37a0-fbe2-4d4e-b627-2c345581be0a None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:12.174185 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 39/76] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:05:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:12.174185 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba37a0-fbe2-4d4e-b627-2c345581be0a None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:12.187784 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:12.193166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6036b7c-631e-4950-92fc-36ad641d2d62 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:12.226209 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a23e35-5391-4362-85c4-2e3d788f368c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:12.226817 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a23e35-5391-4362-85c4-2e3d788f368c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:12.227096 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a23e35-5391-4362-85c4-2e3d788f368c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:12.227365 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a23e35-5391-4362-85c4-2e3d788f368c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:12.275341 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba37a0-fbe2-4d4e-b627-2c345581be0a None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:12.295273 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a23e35-5391-4362-85c4-2e3d788f368c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:12.296780 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a23e35-5391-4362-85c4-2e3d788f368c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:12.309368 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60ef857-4cbc-4262-afb5-8f8487d77218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:12.309999 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ef857-4cbc-4262-afb5-8f8487d77218 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:12.310310 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ef857-4cbc-4262-afb5-8f8487d77218 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:12.310580 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ef857-4cbc-4262-afb5-8f8487d77218 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:12.321279 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60ef857-4cbc-4262-afb5-8f8487d77218 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:12.327110 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-e60ef857-4cbc-4262-afb5-8f8487d77218 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 05:05:12.327110 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:12.327110 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:12.327110 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:12.327110 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:12.327110 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:12.327110 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:12.327110 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:12.327110 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:12.327110 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:12.327110 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:12.327110 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:12.327110 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:12.327110 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:12.327110 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:12.327110 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:12.328142 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:12.328142 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:12.328142 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:12.328142 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:12.328142 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:12.328142 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:12.328142 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:12.328142 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:12.328142 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:12.328142 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:12.328142 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:12.328142 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:12.328142 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:12.328142 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:12.328142 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:12.328142 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:12.329216 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:12.329216 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:12.329216 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:12.329216 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:12.329216 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:12.329216 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:12.329216 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:12.329216 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:12.329216 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:12.329216 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:12.329216 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 05:05:12.329216 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:05:12.329216 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 40/77] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:05:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:05:12.330072 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60ef857-4cbc-4262-afb5-8f8487d77218 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:12.354109 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:12.396502 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:12.401584 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 38/78] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:05:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 05:05:12.403129 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631cf3fc-0a75-4771-802a-85dcc059c6c7 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:12.403636 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631cf3fc-0a75-4771-802a-85dcc059c6c7 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:12.403900 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631cf3fc-0a75-4771-802a-85dcc059c6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:12.421114 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-631cf3fc-0a75-4771-802a-85dcc059c6c7 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:12.422340 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-631cf3fc-0a75-4771-802a-85dcc059c6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:12.430025 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60ef857-4cbc-4262-afb5-8f8487d77218 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:12.463898 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dee854-8ac4-488c-b6c2-7fa368343f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:12.464511 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dee854-8ac4-488c-b6c2-7fa368343f94 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:12.464791 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dee854-8ac4-488c-b6c2-7fa368343f94 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:12.465068 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dee854-8ac4-488c-b6c2-7fa368343f94 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:12.465738 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dee854-8ac4-488c-b6c2-7fa368343f94 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:12.466067 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dee854-8ac4-488c-b6c2-7fa368343f94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:12.467488 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dee854-8ac4-488c-b6c2-7fa368343f94 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:12.475453 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 41/79] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:05:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:12.476049 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dee854-8ac4-488c-b6c2-7fa368343f94 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:12.491891 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 39/80] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:12.492726 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631cf3fc-0a75-4771-802a-85dcc059c6c7 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:12.573452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:12.589718 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dee854-8ac4-488c-b6c2-7fa368343f94 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:12.595875 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631cf3fc-0a75-4771-802a-85dcc059c6c7 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:12.622310 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:12.627601 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f5dde-dd5c-4bf4-9d9b-6dbf36d6f6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:12.628238 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f5dde-dd5c-4bf4-9d9b-6dbf36d6f6c5 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:12.628519 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f5dde-dd5c-4bf4-9d9b-6dbf36d6f6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:12.628789 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f5dde-dd5c-4bf4-9d9b-6dbf36d6f6c5 None admin] PATH_INFO: `/v3/users/c96b7a6c0bc940caab4581734c1919ee` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:12.629896 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980f5dde-dd5c-4bf4-9d9b-6dbf36d6f6c5 None admin] RBAC: Authorizing `identity:get_user(user_id=c96b7a6c0bc940caab4581734c1919ee)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:12.630420 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17dcb73-6606-4137-9520-2632f2d131a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:12.631027 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17dcb73-6606-4137-9520-2632f2d131a7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:12.631304 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17dcb73-6606-4137-9520-2632f2d131a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:12.631449 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980f5dde-dd5c-4bf4-9d9b-6dbf36d6f6c5 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:12.631558 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17dcb73-6606-4137-9520-2632f2d131a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:12.632217 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17dcb73-6606-4137-9520-2632f2d131a7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:12.632514 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17dcb73-6606-4137-9520-2632f2d131a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:12.633255 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 42/81] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:05:12 2022] GET /identity/v3/users/c96b7a6c0bc940caab4581734c1919ee => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:12.633883 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17dcb73-6606-4137-9520-2632f2d131a7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:12.640646 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f5dde-dd5c-4bf4-9d9b-6dbf36d6f6c5 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:12.644913 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 40/82] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 05:05:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:12.645115 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17dcb73-6606-4137-9520-2632f2d131a7 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:12.747472 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17dcb73-6606-4137-9520-2632f2d131a7 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:12.748892 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f5dde-dd5c-4bf4-9d9b-6dbf36d6f6c5 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:12.781401 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a122bc4c-ed1b-4f01-88cf-af3f90de2e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:12.782084 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122bc4c-ed1b-4f01-88cf-af3f90de2e38 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:12.782351 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122bc4c-ed1b-4f01-88cf-af3f90de2e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:12.782590 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef23814-0e0d-4469-bdf2-9f34dd94e793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:12.782749 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122bc4c-ed1b-4f01-88cf-af3f90de2e38 None admin] PATH_INFO: `/v3/users/bbcee328f8db4f5cb50f75830eb1cb5a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:12.783223 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef23814-0e0d-4469-bdf2-9f34dd94e793 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:12.783508 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef23814-0e0d-4469-bdf2-9f34dd94e793 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:12.783786 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef23814-0e0d-4469-bdf2-9f34dd94e793 None admin] PATH_INFO: `/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:12.783979 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a122bc4c-ed1b-4f01-88cf-af3f90de2e38 None admin] RBAC: Authorizing `identity:get_user(user_id=bbcee328f8db4f5cb50f75830eb1cb5a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:12.784907 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef23814-0e0d-4469-bdf2-9f34dd94e793 None admin] RBAC: Authorizing `identity:get_project(project_id=f9d86697e3eb4fd6b8f82a013cfcb5ba)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:12.785445 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a122bc4c-ed1b-4f01-88cf-af3f90de2e38 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:12.786395 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef23814-0e0d-4469-bdf2-9f34dd94e793 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:12.787076 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 41/83] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:05:12 2022] GET /identity/v3/users/bbcee328f8db4f5cb50f75830eb1cb5a => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:12.787076 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a122bc4c-ed1b-4f01-88cf-af3f90de2e38 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:12.791821 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 43/84] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:05:12 2022] GET /identity/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:12.791821 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef23814-0e0d-4469-bdf2-9f34dd94e793 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:12.813351 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:12.848464 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:12.890639 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef23814-0e0d-4469-bdf2-9f34dd94e793 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:12.890800 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a122bc4c-ed1b-4f01-88cf-af3f90de2e38 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:12.923309 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee65e2bd-f395-4911-8ef5-7c61aa22e613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:12.923457 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b80720-03c9-4914-813a-f6d7789abdd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:12.923966 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65e2bd-f395-4911-8ef5-7c61aa22e613 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:12.924058 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b80720-03c9-4914-813a-f6d7789abdd9 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:12.924248 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65e2bd-f395-4911-8ef5-7c61aa22e613 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:12.924338 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b80720-03c9-4914-813a-f6d7789abdd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:12.924524 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65e2bd-f395-4911-8ef5-7c61aa22e613 None admin] PATH_INFO: `/v3/users/bbcee328f8db4f5cb50f75830eb1cb5a` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:12.924609 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b80720-03c9-4914-813a-f6d7789abdd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:12.925656 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee65e2bd-f395-4911-8ef5-7c61aa22e613 None admin] RBAC: Authorizing `identity:get_user(user_id=bbcee328f8db4f5cb50f75830eb1cb5a)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:12.927190 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee65e2bd-f395-4911-8ef5-7c61aa22e613 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:12.929900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 44/85] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:05:12 2022] GET /identity/v3/users/bbcee328f8db4f5cb50f75830eb1cb5a => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:12.931833 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee65e2bd-f395-4911-8ef5-7c61aa22e613 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:12.991741 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b80720-03c9-4914-813a-f6d7789abdd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:12.994177 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b80720-03c9-4914-813a-f6d7789abdd9 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:13.022929 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:13.034845 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee65e2bd-f395-4911-8ef5-7c61aa22e613 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:13.048820 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:13.067574 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a0d5ea-bed2-4f81-93bd-cb6c4450c8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:13.068204 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0d5ea-bed2-4f81-93bd-cb6c4450c8e9 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:13.068493 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0d5ea-bed2-4f81-93bd-cb6c4450c8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:13.068762 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0d5ea-bed2-4f81-93bd-cb6c4450c8e9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:13.069879 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a0d5ea-bed2-4f81-93bd-cb6c4450c8e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:13.071376 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a0d5ea-bed2-4f81-93bd-cb6c4450c8e9 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:13.072957 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 45/86] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:05:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:13.073722 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a0d5ea-bed2-4f81-93bd-cb6c4450c8e9 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:13.095061 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 42/87] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:05:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 05:05:13.096125 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b80720-03c9-4914-813a-f6d7789abdd9 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:13.173735 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a0d5ea-bed2-4f81-93bd-cb6c4450c8e9 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:13.195756 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b80720-03c9-4914-813a-f6d7789abdd9 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:13.206588 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f470140b-de5e-48b4-8437-e5f6a0cebbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:13.207243 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f470140b-de5e-48b4-8437-e5f6a0cebbc2 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:13.207530 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f470140b-de5e-48b4-8437-e5f6a0cebbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:13.207805 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f470140b-de5e-48b4-8437-e5f6a0cebbc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:13.229847 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f72e9-48a0-41ab-96ea-380fd4849f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:13.230486 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f72e9-48a0-41ab-96ea-380fd4849f27 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:13.230768 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f72e9-48a0-41ab-96ea-380fd4849f27 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:13.231038 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f72e9-48a0-41ab-96ea-380fd4849f27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:13.231777 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8f72e9-48a0-41ab-96ea-380fd4849f27 None admin] RBAC: Adding query filter params (role.id=e3235eda7c654e2abf96ed6a494f5152, scope.project.id=f9d86697e3eb4fd6b8f82a013cfcb5ba, user.id=bbcee328f8db4f5cb50f75830eb1cb5a) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:13.232086 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8f72e9-48a0-41ab-96ea-380fd4849f27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:13.232432 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:13.233646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8f72e9-48a0-41ab-96ea-380fd4849f27 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:13.242563 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 43/88] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:05:13 2022] GET /identity/v3/role_assignments?user.id=bbcee328f8db4f5cb50f75830eb1cb5a&scope.project.id=f9d86697e3eb4fd6b8f82a013cfcb5ba&role.id=e3235eda7c654e2abf96ed6a494f5152 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:13.242563 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f72e9-48a0-41ab-96ea-380fd4849f27 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:13.279246 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f470140b-de5e-48b4-8437-e5f6a0cebbc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:13.280842 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f470140b-de5e-48b4-8437-e5f6a0cebbc2 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:13.285720 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:13.382529 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f72e9-48a0-41ab-96ea-380fd4849f27 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:13.399258 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 46/89] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:05:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 05:05:13.399975 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f470140b-de5e-48b4-8437-e5f6a0cebbc2 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:13.426171 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44970169-96f9-4262-9bef-6dd6f346982b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:13.427263 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44970169-96f9-4262-9bef-6dd6f346982b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:13.428071 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44970169-96f9-4262-9bef-6dd6f346982b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:13.428853 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44970169-96f9-4262-9bef-6dd6f346982b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:13.457051 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:13.499996 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f470140b-de5e-48b4-8437-e5f6a0cebbc2 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:13.501870 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44970169-96f9-4262-9bef-6dd6f346982b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:13.503483 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44970169-96f9-4262-9bef-6dd6f346982b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:13.528918 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:13.534964 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c8d2dd-1550-436f-9631-d27961644d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:13.535573 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8d2dd-1550-436f-9631-d27961644d3c None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:13.535848 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8d2dd-1550-436f-9631-d27961644d3c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:13.536115 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8d2dd-1550-436f-9631-d27961644d3c None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:13.536757 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c8d2dd-1550-436f-9631-d27961644d3c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:13.538241 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c8d2dd-1550-436f-9631-d27961644d3c None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:13.566101 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 47/90] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Aug 31 05:05:13 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:05:13.566101 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c8d2dd-1550-436f-9631-d27961644d3c None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:13.633993 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 44/91] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:05:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 05:05:13.634909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44970169-96f9-4262-9bef-6dd6f346982b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:13.706772 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c8d2dd-1550-436f-9631-d27961644d3c None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:13.722406 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:13.749135 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099a4166-5a57-4ebb-b8bf-7894bbc73516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:13.749869 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a4166-5a57-4ebb-b8bf-7894bbc73516 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:13.750198 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a4166-5a57-4ebb-b8bf-7894bbc73516 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:13.750474 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a4166-5a57-4ebb-b8bf-7894bbc73516 None admin] PATH_INFO: `/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:13.754053 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099a4166-5a57-4ebb-b8bf-7894bbc73516 None admin] RBAC: Authorizing `identity:get_project(project_id=f9d86697e3eb4fd6b8f82a013cfcb5ba)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:13.754053 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099a4166-5a57-4ebb-b8bf-7894bbc73516 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:13.755450 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 48/92] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:05:13 2022] GET /identity/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:13.756203 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099a4166-5a57-4ebb-b8bf-7894bbc73516 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:13.765883 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44970169-96f9-4262-9bef-6dd6f346982b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:13.801037 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:13.812511 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade6cc72-0711-4d50-bc61-6364dcd13c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:13.813157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade6cc72-0711-4d50-bc61-6364dcd13c89 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:13.813432 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade6cc72-0711-4d50-bc61-6364dcd13c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:13.813698 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade6cc72-0711-4d50-bc61-6364dcd13c89 None admin] PATH_INFO: `/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:13.814895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade6cc72-0711-4d50-bc61-6364dcd13c89 None admin] RBAC: Authorizing `identity:get_project(project_id=f9d86697e3eb4fd6b8f82a013cfcb5ba)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:13.816356 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade6cc72-0711-4d50-bc61-6364dcd13c89 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:13.818171 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 45/93] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:05:13 2022] GET /identity/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:13.819709 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade6cc72-0711-4d50-bc61-6364dcd13c89 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:13.874736 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099a4166-5a57-4ebb-b8bf-7894bbc73516 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:13.917036 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9080023e-222f-404d-b13a-09f3d757a1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:13.917673 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080023e-222f-404d-b13a-09f3d757a1bf None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:13.917950 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080023e-222f-404d-b13a-09f3d757a1bf None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:13.918421 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080023e-222f-404d-b13a-09f3d757a1bf None admin] PATH_INFO: `/v3/projects/17a2a154edcf47af93f4f542d556d044` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:13.919846 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9080023e-222f-404d-b13a-09f3d757a1bf None admin] RBAC: Authorizing `identity:get_project(project_id=17a2a154edcf47af93f4f542d556d044)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:13.921360 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9080023e-222f-404d-b13a-09f3d757a1bf None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:13.923640 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 49/94] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:05:13 2022] GET /identity/v3/projects/17a2a154edcf47af93f4f542d556d044 => generated 312 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:13.924445 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9080023e-222f-404d-b13a-09f3d757a1bf None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:13.939878 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade6cc72-0711-4d50-bc61-6364dcd13c89 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:13.979716 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e77aa-8f1b-420a-b3f1-f98204d3f9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:13.980895 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:13.981205 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e77aa-8f1b-420a-b3f1-f98204d3f9d4 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:13.982027 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e77aa-8f1b-420a-b3f1-f98204d3f9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:13.982516 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e77aa-8f1b-420a-b3f1-f98204d3f9d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:13.983440 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02e77aa-8f1b-420a-b3f1-f98204d3f9d4 None admin] RBAC: Adding query filter params (role.id=c3d5589dd2864be9a0f26f5802d874cd, scope.project.id=f9d86697e3eb4fd6b8f82a013cfcb5ba, user.id=c96b7a6c0bc940caab4581734c1919ee) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:13.984045 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02e77aa-8f1b-420a-b3f1-f98204d3f9d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:13.985747 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02e77aa-8f1b-420a-b3f1-f98204d3f9d4 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:13.994093 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 46/95] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:05:13 2022] GET /identity/v3/role_assignments?user.id=c96b7a6c0bc940caab4581734c1919ee&scope.project.id=f9d86697e3eb4fd6b8f82a013cfcb5ba&role.id=c3d5589dd2864be9a0f26f5802d874cd => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:13.995665 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e77aa-8f1b-420a-b3f1-f98204d3f9d4 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:14.066629 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:14.093525 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9080023e-222f-404d-b13a-09f3d757a1bf None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:14.158814 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e77aa-8f1b-420a-b3f1-f98204d3f9d4 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:14.162795 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc420e-dccc-4945-b008-428bd483e671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:14.164707 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc420e-dccc-4945-b008-428bd483e671 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:14.165169 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc420e-dccc-4945-b008-428bd483e671 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:14.165881 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc420e-dccc-4945-b008-428bd483e671 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:14.166832 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc420e-dccc-4945-b008-428bd483e671 None admin] RBAC: Adding query filter params (role.id=a74bf01dd66f49f5988a197da269c822, scope.project.id=f9d86697e3eb4fd6b8f82a013cfcb5ba, user.id=bbcee328f8db4f5cb50f75830eb1cb5a) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:14.171921 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc420e-dccc-4945-b008-428bd483e671 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:14.173790 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc420e-dccc-4945-b008-428bd483e671 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:14.183826 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 50/96] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:05:13 2022] GET /identity/v3/role_assignments?user.id=bbcee328f8db4f5cb50f75830eb1cb5a&scope.project.id=f9d86697e3eb4fd6b8f82a013cfcb5ba&role.id=a74bf01dd66f49f5988a197da269c822 => generated 259 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:14.215861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d67376-5b38-4808-b6ff-de1f43bb02c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:14.216842 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d67376-5b38-4808-b6ff-de1f43bb02c5 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:14.217318 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d67376-5b38-4808-b6ff-de1f43bb02c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:14.217755 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d67376-5b38-4808-b6ff-de1f43bb02c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:14.218776 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d67376-5b38-4808-b6ff-de1f43bb02c5 None admin] RBAC: Adding query filter params (role.id=e3235eda7c654e2abf96ed6a494f5152, scope.project.id=17a2a154edcf47af93f4f542d556d044, user.id=bbcee328f8db4f5cb50f75830eb1cb5a) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:14.219255 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d67376-5b38-4808-b6ff-de1f43bb02c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:14.220944 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d67376-5b38-4808-b6ff-de1f43bb02c5 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:14.232207 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 47/97] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:05:13 2022] GET /identity/v3/role_assignments?user.id=bbcee328f8db4f5cb50f75830eb1cb5a&scope.project.id=17a2a154edcf47af93f4f542d556d044&role.id=e3235eda7c654e2abf96ed6a494f5152 => generated 259 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:15.501801 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d196be-fddd-470c-97fc-6fb3e22b8903 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:15.502355 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d196be-fddd-470c-97fc-6fb3e22b8903 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:15.502430 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d196be-fddd-470c-97fc-6fb3e22b8903 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:15.503913 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 51/98] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:15.509535 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b024d5db-9515-4e9a-957a-c90ddefb70f2 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:15.510487 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b024d5db-9515-4e9a-957a-c90ddefb70f2 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:15.510972 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b024d5db-9515-4e9a-957a-c90ddefb70f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:15.539012 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b024d5db-9515-4e9a-957a-c90ddefb70f2 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:15.540565 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b024d5db-9515-4e9a-957a-c90ddefb70f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:15.623628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 48/99] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:16.156125 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d196be-fddd-470c-97fc-6fb3e22b8903 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:16.297801 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d196be-fddd-470c-97fc-6fb3e22b8903 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:16.314230 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:16.342133 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad9a538-e126-4138-933c-81ec0a79752e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:16.342826 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9a538-e126-4138-933c-81ec0a79752e None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:16.343179 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9a538-e126-4138-933c-81ec0a79752e None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:16.343456 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9a538-e126-4138-933c-81ec0a79752e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:16.358889 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad9a538-e126-4138-933c-81ec0a79752e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:16.364277 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-1ad9a538-e126-4138-933c-81ec0a79752e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 05:05:16.364277 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:16.364277 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:16.364277 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:16.364277 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:16.364277 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:16.364277 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:16.364277 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:16.364277 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:16.364277 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:16.364277 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:16.364277 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:16.364277 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:16.364277 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:16.364277 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:16.364277 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:16.365117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:16.365117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:16.365117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:16.365117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:16.365117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:16.365117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:16.365117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:16.365117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:16.365117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:16.365117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:16.365117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:16.365117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:16.365117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:16.365117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:16.365117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:16.365117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:16.365900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:16.365900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:16.365900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:16.365900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:16.365900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:16.365900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:16.365900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:16.365900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:16.365900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:16.365900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:16.365900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 05:05:16.365900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:05:16.365900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 52/100] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:05:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:05:16.372919 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b024d5db-9515-4e9a-957a-c90ddefb70f2 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:16.430072 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164021d7-e126-40d6-a9f3-9d916029c8a9 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:16.430363 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164021d7-e126-40d6-a9f3-9d916029c8a9 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:16.430601 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164021d7-e126-40d6-a9f3-9d916029c8a9 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:16.432283 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 53/101] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:16.437748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:16.438105 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a3b69-8f95-4caa-b367-46cb2db44bac None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:16.439250 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a3b69-8f95-4caa-b367-46cb2db44bac None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:16.439741 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a3b69-8f95-4caa-b367-46cb2db44bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:16.469641 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-a68a3b69-8f95-4caa-b367-46cb2db44bac None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:16.471634 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a68a3b69-8f95-4caa-b367-46cb2db44bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:16.556231 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b024d5db-9515-4e9a-957a-c90ddefb70f2 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:16.584101 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 54/102] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 05:05:16.622027 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090bfe6a-94a2-4d31-9c97-23e4ac32a073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:16.622654 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090bfe6a-94a2-4d31-9c97-23e4ac32a073 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:16.622936 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090bfe6a-94a2-4d31-9c97-23e4ac32a073 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:16.623213 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090bfe6a-94a2-4d31-9c97-23e4ac32a073 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:16.623891 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090bfe6a-94a2-4d31-9c97-23e4ac32a073 None admin] RBAC: Adding query filter params (name=member) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:16.624195 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090bfe6a-94a2-4d31-9c97-23e4ac32a073 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:16.625640 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090bfe6a-94a2-4d31-9c97-23e4ac32a073 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:16.642665 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 49/103] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 05:05:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:16.648050 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a3b69-8f95-4caa-b367-46cb2db44bac None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:16.660485 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f904ae6-3729-403f-9194-fc2464552fbc None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:16.660835 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f904ae6-3729-403f-9194-fc2464552fbc None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:16.661039 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f904ae6-3729-403f-9194-fc2464552fbc None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:16.662553 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 50/104] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 05:05:16.668054 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7171ad06-d0f8-46d8-96a8-7f5057a9d958 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:16.668342 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7171ad06-d0f8-46d8-96a8-7f5057a9d958 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:16.668621 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7171ad06-d0f8-46d8-96a8-7f5057a9d958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:16.696288 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-7171ad06-d0f8-46d8-96a8-7f5057a9d958 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:16.697409 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7171ad06-d0f8-46d8-96a8-7f5057a9d958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:16.813558 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 51/105] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 05:05:16.821503 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a3b69-8f95-4caa-b367-46cb2db44bac None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:16.842508 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:16.857932 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423ebf28-e4a1-4639-b7d5-6a2f5f0b3aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:16.858637 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ebf28-e4a1-4639-b7d5-6a2f5f0b3aef None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:16.858924 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ebf28-e4a1-4639-b7d5-6a2f5f0b3aef None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:16.859198 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ebf28-e4a1-4639-b7d5-6a2f5f0b3aef None admin] PATH_INFO: `/v3/users/bbcee328f8db4f5cb50f75830eb1cb5a` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:16.860355 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423ebf28-e4a1-4639-b7d5-6a2f5f0b3aef None admin] RBAC: Authorizing `identity:get_user(user_id=bbcee328f8db4f5cb50f75830eb1cb5a)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:16.861819 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423ebf28-e4a1-4639-b7d5-6a2f5f0b3aef None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:16.863692 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 55/106] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:05:16 2022] GET /identity/v3/users/bbcee328f8db4f5cb50f75830eb1cb5a => generated 275 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:16.869710 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7171ad06-d0f8-46d8-96a8-7f5057a9d958 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:16.924018 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:16.992151 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7171ad06-d0f8-46d8-96a8-7f5057a9d958 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:17.034258 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6adcd6-76fd-4ef2-affb-a62da5ed3cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:17.034955 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6adcd6-76fd-4ef2-affb-a62da5ed3cd3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:17.035236 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6adcd6-76fd-4ef2-affb-a62da5ed3cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:17.035562 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6adcd6-76fd-4ef2-affb-a62da5ed3cd3 None admin] PATH_INFO: `/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:17.036747 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6adcd6-76fd-4ef2-affb-a62da5ed3cd3 None admin] RBAC: Authorizing `identity:get_project(project_id=f9d86697e3eb4fd6b8f82a013cfcb5ba)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:17.038293 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6adcd6-76fd-4ef2-affb-a62da5ed3cd3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:17.039957 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 52/107] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:05:16 2022] GET /identity/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:17.045959 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423ebf28-e4a1-4639-b7d5-6a2f5f0b3aef None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:17.118476 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01148b-de7f-4848-9cbc-f6a238301a15 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:17.118798 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01148b-de7f-4848-9cbc-f6a238301a15 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:17.119014 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01148b-de7f-4848-9cbc-f6a238301a15 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:17.120597 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 53/108] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 05:05:17.134751 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e84e9-abbc-477b-b812-4190d7fb8a02 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:17.135036 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e84e9-abbc-477b-b812-4190d7fb8a02 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:17.135371 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e84e9-abbc-477b-b812-4190d7fb8a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:17.160710 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-688e84e9-abbc-477b-b812-4190d7fb8a02 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:17.161879 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-688e84e9-abbc-477b-b812-4190d7fb8a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:17.174394 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423ebf28-e4a1-4639-b7d5-6a2f5f0b3aef None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:17.210865 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03233db-06a6-4722-8432-af425f9b9e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:17.211651 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03233db-06a6-4722-8432-af425f9b9e85 None admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:17.211933 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03233db-06a6-4722-8432-af425f9b9e85 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:17.212313 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03233db-06a6-4722-8432-af425f9b9e85 None admin] PATH_INFO: `/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba/users/bbcee328f8db4f5cb50f75830eb1cb5a/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:17.214277 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03233db-06a6-4722-8432-af425f9b9e85 None admin] RBAC: Authorizing `identity:create_grant(project_id=f9d86697e3eb4fd6b8f82a013cfcb5ba, user_id=bbcee328f8db4f5cb50f75830eb1cb5a, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:17.215962 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03233db-06a6-4722-8432-af425f9b9e85 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:17.232834 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 56/109] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 05:05:17 2022] PUT /identity/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba/users/bbcee328f8db4f5cb50f75830eb1cb5a/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:05:17.234252 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c81438-ab4d-44b5-b87f-43d64baa986b None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:17.234504 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c81438-ab4d-44b5-b87f-43d64baa986b None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:17.234785 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c81438-ab4d-44b5-b87f-43d64baa986b None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:17.236073 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 57/110] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:17.236286 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 54/111] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 05:05:17.240546 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c81438-ab4d-44b5-b87f-43d64baa986b None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:17.244941 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dada7f-7988-421a-986d-5d4ba95c2e65 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:17.246976 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dada7f-7988-421a-986d-5d4ba95c2e65 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:17.247896 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dada7f-7988-421a-986d-5d4ba95c2e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:17.287836 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f2dada7f-7988-421a-986d-5d4ba95c2e65 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:17.290292 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f2dada7f-7988-421a-986d-5d4ba95c2e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:17.307946 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:17.334413 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:17.376843 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 55/112] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 05:05:17.386297 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186fea7-7a0a-466c-ba27-390056ead65c None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:17.386590 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186fea7-7a0a-466c-ba27-390056ead65c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:17.386994 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186fea7-7a0a-466c-ba27-390056ead65c None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:17.388865 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 56/113] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:17 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:17.389773 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f186fea7-7a0a-466c-ba27-390056ead65c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:17.416571 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c81438-ab4d-44b5-b87f-43d64baa986b None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:17.456486 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74917178-e764-42c9-a7f0-69ad324d3e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:17.457227 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74917178-e764-42c9-a7f0-69ad324d3e8f None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:17.457569 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74917178-e764-42c9-a7f0-69ad324d3e8f None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:17.457908 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74917178-e764-42c9-a7f0-69ad324d3e8f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:17.470997 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74917178-e764-42c9-a7f0-69ad324d3e8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:17.477867 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-74917178-e764-42c9-a7f0-69ad324d3e8f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 05:05:17.477867 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:17.477867 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:17.477867 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:17.477867 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:17.477867 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:17.477867 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:17.477867 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:17.477867 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:17.477867 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:17.477867 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:17.477867 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:17.477867 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:17.477867 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:17.477867 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:17.477867 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:17.478748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:17.478748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:17.478748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:17.478748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:17.478748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:17.478748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:17.478748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:17.478748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:17.478748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:17.478748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:17.478748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:17.478748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:17.478748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:17.478748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:17.478748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:17.478748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:17.479505 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:17.479505 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:17.479505 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:17.479505 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:17.479505 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:17.479505 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:17.479505 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:17.479505 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:17.479505 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:17.479505 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:17.479505 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 05:05:17.479505 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:05:17.481927 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 58/114] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:05:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:05:17.484051 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff025a6-f5d6-4671-9886-3cf456716fc4 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:17.484493 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff025a6-f5d6-4671-9886-3cf456716fc4 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:17.484931 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff025a6-f5d6-4671-9886-3cf456716fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:17.504091 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-6ff025a6-f5d6-4671-9886-3cf456716fc4 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:17.505713 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-6ff025a6-f5d6-4671-9886-3cf456716fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:17.524544 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f186fea7-7a0a-466c-ba27-390056ead65c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:17.574107 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66140cf-577e-4223-80c4-44b2b7b7adca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:17.575537 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66140cf-577e-4223-80c4-44b2b7b7adca None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:17.576107 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66140cf-577e-4223-80c4-44b2b7b7adca None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:17.576569 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66140cf-577e-4223-80c4-44b2b7b7adca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:17.582690 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 59/115] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:17.586102 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff025a6-f5d6-4671-9886-3cf456716fc4 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:17.598634 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66140cf-577e-4223-80c4-44b2b7b7adca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:17.603756 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-a66140cf-577e-4223-80c4-44b2b7b7adca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:05:17.603756 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:17.603756 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:17.603756 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:17.603756 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:17.603756 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:17.603756 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:17.603756 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:17.603756 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:17.603756 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:17.603756 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:17.603756 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:17.603756 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:17.603756 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:17.603756 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:17.603756 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:17.604591 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:17.604591 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:17.604591 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:17.604591 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:17.604591 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:17.604591 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:17.604591 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:17.604591 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:17.604591 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:17.604591 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:17.604591 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:17.604591 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:17.604591 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:17.604591 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:17.604591 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:17.604591 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:17.605342 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:17.605342 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:17.605342 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:17.605342 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:17.605342 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:17.605342 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:17.605342 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:17.605342 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:17.605342 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:17.605342 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:17.605342 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:05:17.605342 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:05:17.612387 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 57/116] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 05:05:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:05:17.613471 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66140cf-577e-4223-80c4-44b2b7b7adca None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:17.641768 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:17.662681 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:17.740765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff025a6-f5d6-4671-9886-3cf456716fc4 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:17.774780 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66140cf-577e-4223-80c4-44b2b7b7adca None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:17.776316 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571cfba8-f3d2-408c-be3b-a877159c5db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:17.776937 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571cfba8-f3d2-408c-be3b-a877159c5db7 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:17.777221 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571cfba8-f3d2-408c-be3b-a877159c5db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:17.777495 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571cfba8-f3d2-408c-be3b-a877159c5db7 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:17.778286 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571cfba8-f3d2-408c-be3b-a877159c5db7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:17.778602 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571cfba8-f3d2-408c-be3b-a877159c5db7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:17.780015 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571cfba8-f3d2-408c-be3b-a877159c5db7 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:17.789042 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 60/117] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 05:05:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:17.789863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571cfba8-f3d2-408c-be3b-a877159c5db7 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:17.817414 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb32562-2d71-4d82-ab37-0a67da9bc41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:17.818187 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb32562-2d71-4d82-ab37-0a67da9bc41c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:17.818465 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb32562-2d71-4d82-ab37-0a67da9bc41c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:17.818728 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb32562-2d71-4d82-ab37-0a67da9bc41c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:17.830277 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb32562-2d71-4d82-ab37-0a67da9bc41c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:17.835390 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-0cb32562-2d71-4d82-ab37-0a67da9bc41c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 05:05:17.835390 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:17.835390 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:17.835390 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:17.835390 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:17.835390 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:17.835390 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:17.835390 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:17.835390 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:17.835390 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:17.835390 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:17.835390 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:17.835390 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:17.835390 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:17.835390 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:17.835390 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:17.836240 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:17.836240 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:17.836240 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:17.836240 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:17.836240 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:17.836240 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:17.836240 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:17.836240 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:17.836240 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:17.836240 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:17.836240 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:17.836240 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:17.836240 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:17.836240 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:17.836240 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:17.836240 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:17.837396 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:17.837396 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:17.837396 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:17.837396 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:17.837396 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:17.837396 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:17.837396 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:17.837396 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:17.837396 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:17.837396 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:17.837396 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 05:05:17.837396 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:05:17.837396 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 58/118] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 05:05:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:05:17.838421 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb32562-2d71-4d82-ab37-0a67da9bc41c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:17.873428 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:17.892242 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571cfba8-f3d2-408c-be3b-a877159c5db7 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:17.928995 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee43a9-efdc-4bd8-9bc5-3533fa412c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:17.929811 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee43a9-efdc-4bd8-9bc5-3533fa412c47 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:17.929918 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee43a9-efdc-4bd8-9bc5-3533fa412c47 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:17.930170 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee43a9-efdc-4bd8-9bc5-3533fa412c47 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:17.930831 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee43a9-efdc-4bd8-9bc5-3533fa412c47 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:17.931131 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee43a9-efdc-4bd8-9bc5-3533fa412c47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:17.932513 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee43a9-efdc-4bd8-9bc5-3533fa412c47 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:17.933487 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:17.942078 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 61/119] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:05:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:17.942901 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee43a9-efdc-4bd8-9bc5-3533fa412c47 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:17.953893 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb32562-2d71-4d82-ab37-0a67da9bc41c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:17.992183 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b15ffb-3411-47af-b512-329935e87de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:17.992934 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b15ffb-3411-47af-b512-329935e87de4 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:17.993087 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b15ffb-3411-47af-b512-329935e87de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:17.993679 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b15ffb-3411-47af-b512-329935e87de4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:17.994821 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b15ffb-3411-47af-b512-329935e87de4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:17.996281 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b15ffb-3411-47af-b512-329935e87de4 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:18.002468 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 59/120] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:05:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:18.002468 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b15ffb-3411-47af-b512-329935e87de4 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:18.057757 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee43a9-efdc-4bd8-9bc5-3533fa412c47 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:18.095380 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d5cc1-0b21-48e4-8eb5-621fef594c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:18.096031 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d5cc1-0b21-48e4-8eb5-621fef594c74 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:18.096301 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d5cc1-0b21-48e4-8eb5-621fef594c74 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:18.096537 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d5cc1-0b21-48e4-8eb5-621fef594c74 None admin] PATH_INFO: `/v3/users/02e7eb819c754f4b9c880d300393dd5c` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:18.110288 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b15ffb-3411-47af-b512-329935e87de4 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:18.112499 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9d5cc1-0b21-48e4-8eb5-621fef594c74 None admin] RBAC: Authorizing `identity:get_user(user_id=02e7eb819c754f4b9c880d300393dd5c)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:18.114222 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9d5cc1-0b21-48e4-8eb5-621fef594c74 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:18.115881 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 62/121] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:05:17 2022] GET /identity/v3/users/02e7eb819c754f4b9c880d300393dd5c => generated 289 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:18.120945 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d5cc1-0b21-48e4-8eb5-621fef594c74 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:18.120945 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:18.150323 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65b5f6-3bb5-4960-9b59-f87c93b152fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:18.150939 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65b5f6-3bb5-4960-9b59-f87c93b152fa None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:18.151219 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65b5f6-3bb5-4960-9b59-f87c93b152fa None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:18.151491 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65b5f6-3bb5-4960-9b59-f87c93b152fa None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:18.152161 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a65b5f6-3bb5-4960-9b59-f87c93b152fa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:18.152468 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a65b5f6-3bb5-4960-9b59-f87c93b152fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:18.153872 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a65b5f6-3bb5-4960-9b59-f87c93b152fa None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:18.166691 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 60/122] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:05:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:18.166691 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65b5f6-3bb5-4960-9b59-f87c93b152fa None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:18.186998 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:18.226396 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d5cc1-0b21-48e4-8eb5-621fef594c74 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:18.261658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b2753-961a-472f-a996-2fd539a6ce18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:18.262323 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b2753-961a-472f-a996-2fd539a6ce18 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:18.262604 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b2753-961a-472f-a996-2fd539a6ce18 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:18.262879 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b2753-961a-472f-a996-2fd539a6ce18 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:18.271968 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65b5f6-3bb5-4960-9b59-f87c93b152fa None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:18.273039 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b2753-961a-472f-a996-2fd539a6ce18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:18.277851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-3c6b2753-961a-472f-a996-2fd539a6ce18 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 05:05:18.277851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:18.277851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:18.277851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:18.277851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:18.277851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:18.277851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:18.277851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:18.277851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:18.277851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:18.277851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:18.277851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:18.277851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:18.277851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:18.277851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:18.277851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:18.278824 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:18.278824 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:18.278824 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:18.278824 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:18.278824 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:18.278824 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:18.278824 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:18.278824 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:18.278824 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:18.278824 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:18.278824 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:18.278824 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:18.278824 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:18.278824 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:18.278824 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:18.278824 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:18.279609 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:18.279609 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:18.279609 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:18.279609 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:18.279609 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:18.279609 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:18.279609 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:18.279609 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:18.279609 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:18.279609 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:18.279609 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 05:05:18.279609 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:05:18.279609 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 63/123] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:05:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:05:18.280232 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b2753-961a-472f-a996-2fd539a6ce18 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:18.310763 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315b7909-3dff-43a7-9367-65ec57991a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:18.311402 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b7909-3dff-43a7-9367-65ec57991a1b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:18.311680 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b7909-3dff-43a7-9367-65ec57991a1b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:18.312053 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b7909-3dff-43a7-9367-65ec57991a1b None admin] PATH_INFO: `/v3/users/c96b7a6c0bc940caab4581734c1919ee` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:18.313204 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315b7909-3dff-43a7-9367-65ec57991a1b None admin] RBAC: Authorizing `identity:get_user(user_id=c96b7a6c0bc940caab4581734c1919ee)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:18.314719 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315b7909-3dff-43a7-9367-65ec57991a1b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:18.316493 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 61/124] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:05:18 2022] GET /identity/v3/users/c96b7a6c0bc940caab4581734c1919ee => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:18.321461 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315b7909-3dff-43a7-9367-65ec57991a1b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:18.342876 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:18.387258 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b2753-961a-472f-a996-2fd539a6ce18 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:18.412634 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:18.423214 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69866711-b1c1-40dc-856e-a97a30001ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:18.423825 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69866711-b1c1-40dc-856e-a97a30001ee1 None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:18.424101 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69866711-b1c1-40dc-856e-a97a30001ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:18.424373 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69866711-b1c1-40dc-856e-a97a30001ee1 None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:18.425031 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69866711-b1c1-40dc-856e-a97a30001ee1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:18.426535 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69866711-b1c1-40dc-856e-a97a30001ee1 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:18.430806 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315b7909-3dff-43a7-9367-65ec57991a1b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:18.462043 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 64/125] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Aug 31 05:05:18 2022] POST /identity/v3/projects => generated 302 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:05:18.462043 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69866711-b1c1-40dc-856e-a97a30001ee1 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:18.479882 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201a758-74a8-4b8e-a231-a7b10a01d7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:18.480784 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201a758-74a8-4b8e-a231-a7b10a01d7ca None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:18.481127 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201a758-74a8-4b8e-a231-a7b10a01d7ca None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:18.481411 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201a758-74a8-4b8e-a231-a7b10a01d7ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:18.575936 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1201a758-74a8-4b8e-a231-a7b10a01d7ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:18.577599 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1201a758-74a8-4b8e-a231-a7b10a01d7ca None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:18.578122 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69866711-b1c1-40dc-856e-a97a30001ee1 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:18.593814 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:18.616947 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9424f2-6edc-4183-b6fe-ed2fc5eabe69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:18.617748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9424f2-6edc-4183-b6fe-ed2fc5eabe69 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:18.618425 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9424f2-6edc-4183-b6fe-ed2fc5eabe69 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:18.618916 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9424f2-6edc-4183-b6fe-ed2fc5eabe69 None admin] PATH_INFO: `/v3/users/bbcee328f8db4f5cb50f75830eb1cb5a` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:18.620581 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9424f2-6edc-4183-b6fe-ed2fc5eabe69 None admin] RBAC: Authorizing `identity:get_user(user_id=bbcee328f8db4f5cb50f75830eb1cb5a)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:18.622274 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9424f2-6edc-4183-b6fe-ed2fc5eabe69 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:18.623874 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 65/126] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:05:18 2022] GET /identity/v3/users/bbcee328f8db4f5cb50f75830eb1cb5a => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:18.625717 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9424f2-6edc-4183-b6fe-ed2fc5eabe69 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:18.688437 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:18.708000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 62/127] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:05:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 05:05:18.709074 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201a758-74a8-4b8e-a231-a7b10a01d7ca None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:18.758139 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9424f2-6edc-4183-b6fe-ed2fc5eabe69 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:18.792765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c70795c-701a-49d6-9a09-c9517c5db37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:18.793383 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70795c-701a-49d6-9a09-c9517c5db37e None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:18.793664 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70795c-701a-49d6-9a09-c9517c5db37e None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:18.793937 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70795c-701a-49d6-9a09-c9517c5db37e None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:18.794664 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c70795c-701a-49d6-9a09-c9517c5db37e None admin] RBAC: Adding query filter params (name=member) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:18.794969 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c70795c-701a-49d6-9a09-c9517c5db37e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:18.796539 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c70795c-701a-49d6-9a09-c9517c5db37e None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:18.805760 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 66/128] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 05:05:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:18.806696 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c70795c-701a-49d6-9a09-c9517c5db37e None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:18.822788 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201a758-74a8-4b8e-a231-a7b10a01d7ca None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:18.856165 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:18.860748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ecc9cb-0e5b-45ee-bf81-fe3db3d8742c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:18.861859 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ecc9cb-0e5b-45ee-bf81-fe3db3d8742c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:18.862316 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ecc9cb-0e5b-45ee-bf81-fe3db3d8742c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:18.862737 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ecc9cb-0e5b-45ee-bf81-fe3db3d8742c None admin] PATH_INFO: `/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:18.864011 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ecc9cb-0e5b-45ee-bf81-fe3db3d8742c None admin] RBAC: Authorizing `identity:get_project(project_id=f9d86697e3eb4fd6b8f82a013cfcb5ba)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:18.866389 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ecc9cb-0e5b-45ee-bf81-fe3db3d8742c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:18.869162 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 63/129] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:05:18 2022] GET /identity/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:18.870312 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ecc9cb-0e5b-45ee-bf81-fe3db3d8742c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:18.920812 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c70795c-701a-49d6-9a09-c9517c5db37e None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:18.938648 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:18.959136 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58954388-c513-4ba1-a5b2-a0a0c107fb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:18.959783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58954388-c513-4ba1-a5b2-a0a0c107fb85 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:18.960067 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58954388-c513-4ba1-a5b2-a0a0c107fb85 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:18.960339 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58954388-c513-4ba1-a5b2-a0a0c107fb85 None admin] PATH_INFO: `/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:18.961474 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58954388-c513-4ba1-a5b2-a0a0c107fb85 None admin] RBAC: Authorizing `identity:get_project(project_id=f9d86697e3eb4fd6b8f82a013cfcb5ba)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:18.963097 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58954388-c513-4ba1-a5b2-a0a0c107fb85 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:18.965024 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 67/130] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:05:18 2022] GET /identity/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:18.965687 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58954388-c513-4ba1-a5b2-a0a0c107fb85 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:18.995482 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ecc9cb-0e5b-45ee-bf81-fe3db3d8742c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:19.038403 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e121846-0ac4-42a8-8027-375442caed84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:19.039481 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e121846-0ac4-42a8-8027-375442caed84 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:19.040171 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e121846-0ac4-42a8-8027-375442caed84 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:19.040855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e121846-0ac4-42a8-8027-375442caed84 None admin] PATH_INFO: `/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:19.042314 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e121846-0ac4-42a8-8027-375442caed84 None admin] RBAC: Authorizing `identity:get_project(project_id=f9d86697e3eb4fd6b8f82a013cfcb5ba)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:19.044477 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e121846-0ac4-42a8-8027-375442caed84 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:19.047045 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 64/131] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:05:18 2022] GET /identity/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:19.047976 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e121846-0ac4-42a8-8027-375442caed84 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:19.089549 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58954388-c513-4ba1-a5b2-a0a0c107fb85 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:19.103347 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:19.136550 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7960c0-84b3-4d5e-8138-25dcc2439b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:19.136550 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7960c0-84b3-4d5e-8138-25dcc2439b67 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:19.136550 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7960c0-84b3-4d5e-8138-25dcc2439b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:19.136550 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7960c0-84b3-4d5e-8138-25dcc2439b67 None admin] PATH_INFO: `/v3/users/bbcee328f8db4f5cb50f75830eb1cb5a` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:19.136967 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7960c0-84b3-4d5e-8138-25dcc2439b67 None admin] RBAC: Authorizing `identity:get_user(user_id=bbcee328f8db4f5cb50f75830eb1cb5a)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:19.136967 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7960c0-84b3-4d5e-8138-25dcc2439b67 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:19.138439 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 68/132] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:05:18 2022] GET /identity/v3/users/bbcee328f8db4f5cb50f75830eb1cb5a => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:19.139312 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7960c0-84b3-4d5e-8138-25dcc2439b67 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:19.164548 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e121846-0ac4-42a8-8027-375442caed84 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:19.192688 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:19.206638 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b0b067-47e0-4f54-9a64-af3629c31e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:19.207547 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b0b067-47e0-4f54-9a64-af3629c31e0c None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:19.207841 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b0b067-47e0-4f54-9a64-af3629c31e0c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:19.208118 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b0b067-47e0-4f54-9a64-af3629c31e0c None admin] PATH_INFO: `/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba/users/c96b7a6c0bc940caab4581734c1919ee/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:19.210036 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b0b067-47e0-4f54-9a64-af3629c31e0c None admin] RBAC: Authorizing `identity:create_grant(project_id=f9d86697e3eb4fd6b8f82a013cfcb5ba, user_id=c96b7a6c0bc940caab4581734c1919ee, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:19.211654 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b0b067-47e0-4f54-9a64-af3629c31e0c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:19.232917 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 65/133] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 05:05:19 2022] PUT /identity/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba/users/c96b7a6c0bc940caab4581734c1919ee/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:05:19.234328 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b0b067-47e0-4f54-9a64-af3629c31e0c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:19.265017 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7960c0-84b3-4d5e-8138-25dcc2439b67 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:19.338043 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b0c30-3aed-4a23-ae73-f67dc3ff422f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:19.338735 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b0c30-3aed-4a23-ae73-f67dc3ff422f None admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:19.339012 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b0c30-3aed-4a23-ae73-f67dc3ff422f None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:19.339285 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b0c30-3aed-4a23-ae73-f67dc3ff422f None admin] PATH_INFO: `/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba/users/bbcee328f8db4f5cb50f75830eb1cb5a/roles/a74bf01dd66f49f5988a197da269c822` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:19.341484 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0b0c30-3aed-4a23-ae73-f67dc3ff422f None admin] RBAC: Authorizing `identity:create_grant(project_id=f9d86697e3eb4fd6b8f82a013cfcb5ba, user_id=bbcee328f8db4f5cb50f75830eb1cb5a, role_id=a74bf01dd66f49f5988a197da269c822)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:19.343128 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0b0c30-3aed-4a23-ae73-f67dc3ff422f None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:19.363373 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 69/134] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 05:05:19 2022] PUT /identity/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba/users/bbcee328f8db4f5cb50f75830eb1cb5a/roles/a74bf01dd66f49f5988a197da269c822 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:05:19.365645 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b0c30-3aed-4a23-ae73-f67dc3ff422f None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:19.414538 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b0b067-47e0-4f54-9a64-af3629c31e0c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:19.428173 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:19.458103 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9d120-30e7-4fa4-bab2-407c11283e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:19.458786 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9d120-30e7-4fa4-bab2-407c11283e02 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:19.459146 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9d120-30e7-4fa4-bab2-407c11283e02 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:19.459467 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9d120-30e7-4fa4-bab2-407c11283e02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:19.460218 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c9d120-30e7-4fa4-bab2-407c11283e02 None admin] RBAC: Adding query filter params (role.id=c87ef585cc814d919ef69ba258ecbbbd, scope.project.id=f9d86697e3eb4fd6b8f82a013cfcb5ba, user.id=02e7eb819c754f4b9c880d300393dd5c) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:19.460520 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c9d120-30e7-4fa4-bab2-407c11283e02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:19.462902 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c9d120-30e7-4fa4-bab2-407c11283e02 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:19.469875 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 66/135] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:05:19 2022] GET /identity/v3/role_assignments?user.id=02e7eb819c754f4b9c880d300393dd5c&scope.project.id=f9d86697e3eb4fd6b8f82a013cfcb5ba&role.id=c87ef585cc814d919ef69ba258ecbbbd => generated 259 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:19.471534 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bfe2e-455c-4a48-9030-8bc88c04f75d None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:19.471818 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bfe2e-455c-4a48-9030-8bc88c04f75d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:19.472089 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bfe2e-455c-4a48-9030-8bc88c04f75d None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:19.475080 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 67/136] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:19.501100 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da1ced-6810-48af-846d-5f771b3be54f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:19.501539 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da1ced-6810-48af-846d-5f771b3be54f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:19.501915 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da1ced-6810-48af-846d-5f771b3be54f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:19.514641 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b0c30-3aed-4a23-ae73-f67dc3ff422f None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:19.547298 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-21da1ced-6810-48af-846d-5f771b3be54f None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:19.548584 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-21da1ced-6810-48af-846d-5f771b3be54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:19.558418 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02a6ed-5c48-453e-8bba-a648059ab7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:19.559311 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02a6ed-5c48-453e-8bba-a648059ab7ec None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:19.559751 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02a6ed-5c48-453e-8bba-a648059ab7ec None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:19.560151 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02a6ed-5c48-453e-8bba-a648059ab7ec None admin] PATH_INFO: `/v3/projects/17a2a154edcf47af93f4f542d556d044` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:19.561466 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb02a6ed-5c48-453e-8bba-a648059ab7ec None admin] RBAC: Authorizing `identity:get_project(project_id=17a2a154edcf47af93f4f542d556d044)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:19.564004 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb02a6ed-5c48-453e-8bba-a648059ab7ec None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:19.566231 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 70/137] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:05:19 2022] GET /identity/v3/projects/17a2a154edcf47af93f4f542d556d044 => generated 312 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:19.572983 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02a6ed-5c48-453e-8bba-a648059ab7ec None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:19.701040 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02a6ed-5c48-453e-8bba-a648059ab7ec None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:19.718193 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:19.722725 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:19.767032 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc10870-7a86-421c-8dd8-e823f929aedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:19.767725 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc10870-7a86-421c-8dd8-e823f929aedd None admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:19.770317 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc10870-7a86-421c-8dd8-e823f929aedd None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:19.770317 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc10870-7a86-421c-8dd8-e823f929aedd None admin] PATH_INFO: `/v3/projects/17a2a154edcf47af93f4f542d556d044/users/bbcee328f8db4f5cb50f75830eb1cb5a/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:19.771873 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc10870-7a86-421c-8dd8-e823f929aedd None admin] RBAC: Authorizing `identity:create_grant(project_id=17a2a154edcf47af93f4f542d556d044, user_id=bbcee328f8db4f5cb50f75830eb1cb5a, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:19.773341 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc10870-7a86-421c-8dd8-e823f929aedd None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:19.789840 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 68/138] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:19.812334 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 71/139] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 05:05:19 2022] PUT /identity/v3/projects/17a2a154edcf47af93f4f542d556d044/users/bbcee328f8db4f5cb50f75830eb1cb5a/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:05:20.236010 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da1ced-6810-48af-846d-5f771b3be54f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:20.358598 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da1ced-6810-48af-846d-5f771b3be54f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:20.406316 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c02f7e-d931-478e-acd5-7facc91305c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:20.407344 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c02f7e-d931-478e-acd5-7facc91305c9 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:20.407814 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c02f7e-d931-478e-acd5-7facc91305c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:20.408272 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c02f7e-d931-478e-acd5-7facc91305c9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:20.419078 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c02f7e-d931-478e-acd5-7facc91305c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:20.424302 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-12c02f7e-d931-478e-acd5-7facc91305c9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 05:05:20.424302 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:20.424302 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:20.424302 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:20.424302 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:20.424302 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:20.424302 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:20.424302 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:20.424302 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:20.424302 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:20.424302 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:20.424302 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:20.424302 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:20.424302 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:20.424302 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:20.424302 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:20.425157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:20.425157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:20.425157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:20.425157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:20.425157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:20.425157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:20.425157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:20.425157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:20.425157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:20.425157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:20.425157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:20.425157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:20.425157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:20.425157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:20.425157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:20.425157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:20.425924 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:20.425924 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:20.425924 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:20.425924 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:20.425924 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:20.425924 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:20.425924 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:20.425924 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:20.425924 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:20.425924 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:20.425924 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 05:05:20.425924 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:05:20.428491 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 69/140] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:05:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:05:20.434452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc10870-7a86-421c-8dd8-e823f929aedd None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:20.549918 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc10870-7a86-421c-8dd8-e823f929aedd None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:20.586830 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e2622a-45e5-4e5e-9e1d-cb8cb709f277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:20.587438 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2622a-45e5-4e5e-9e1d-cb8cb709f277 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:20.587710 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2622a-45e5-4e5e-9e1d-cb8cb709f277 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:20.587976 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2622a-45e5-4e5e-9e1d-cb8cb709f277 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:20.588638 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e2622a-45e5-4e5e-9e1d-cb8cb709f277 None admin] RBAC: Adding query filter params (name=member) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:20.588931 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e2622a-45e5-4e5e-9e1d-cb8cb709f277 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:20.590333 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e2622a-45e5-4e5e-9e1d-cb8cb709f277 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:20.599330 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 72/141] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 05:05:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:20.604884 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c02f7e-d931-478e-acd5-7facc91305c9 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:20.652411 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:20.721056 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c02f7e-d931-478e-acd5-7facc91305c9 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:20.755521 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a6210-c630-4268-a6e0-a64a612c9835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:20.756166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a6210-c630-4268-a6e0-a64a612c9835 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:20.756447 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a6210-c630-4268-a6e0-a64a612c9835 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:20.756720 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a6210-c630-4268-a6e0-a64a612c9835 None admin] PATH_INFO: `/v3/users/bbcee328f8db4f5cb50f75830eb1cb5a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:20.757875 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4a6210-c630-4268-a6e0-a64a612c9835 None admin] RBAC: Authorizing `identity:get_user(user_id=bbcee328f8db4f5cb50f75830eb1cb5a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:20.759379 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4a6210-c630-4268-a6e0-a64a612c9835 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:20.761270 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 70/142] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:05:20 2022] GET /identity/v3/users/bbcee328f8db4f5cb50f75830eb1cb5a => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:20.766738 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e2622a-45e5-4e5e-9e1d-cb8cb709f277 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:20.809315 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:20.896136 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e2622a-45e5-4e5e-9e1d-cb8cb709f277 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:20.919040 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ba413-826b-4fad-a735-29613511c290 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:20.919348 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ba413-826b-4fad-a735-29613511c290 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:20.919604 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ba413-826b-4fad-a735-29613511c290 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:20.921150 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 71/143] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:20.925825 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9297d-b8e9-4828-acc8-ce05227662f6 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:20.926172 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9297d-b8e9-4828-acc8-ce05227662f6 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:20.926449 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9297d-b8e9-4828-acc8-ce05227662f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:20.943679 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219bebc4-b78f-4582-9eab-d3bc92151e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:20.944295 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219bebc4-b78f-4582-9eab-d3bc92151e50 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:20.944570 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219bebc4-b78f-4582-9eab-d3bc92151e50 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:20.944835 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219bebc4-b78f-4582-9eab-d3bc92151e50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:20.962108 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-0dd9297d-b8e9-4828-acc8-ce05227662f6 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:20.963437 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0dd9297d-b8e9-4828-acc8-ce05227662f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:21.043080 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219bebc4-b78f-4582-9eab-d3bc92151e50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:21.044551 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219bebc4-b78f-4582-9eab-d3bc92151e50 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:21.094438 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:21.151664 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 72/144] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:21.226748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 73/145] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:05:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 05:05:21.232380 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd9297d-b8e9-4828-acc8-ce05227662f6 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:21.297188 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:21.386764 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd9297d-b8e9-4828-acc8-ce05227662f6 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:21.433407 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a5edcf-d7d8-4a3a-aeeb-ce7e844bfe0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:21.434060 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a5edcf-d7d8-4a3a-aeeb-ce7e844bfe0a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:21.434341 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a5edcf-d7d8-4a3a-aeeb-ce7e844bfe0a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:21.434608 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a5edcf-d7d8-4a3a-aeeb-ce7e844bfe0a None admin] PATH_INFO: `/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:21.435723 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a5edcf-d7d8-4a3a-aeeb-ce7e844bfe0a None admin] RBAC: Authorizing `identity:get_project(project_id=f9d86697e3eb4fd6b8f82a013cfcb5ba)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:21.437279 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a5edcf-d7d8-4a3a-aeeb-ce7e844bfe0a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:21.439929 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 73/146] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:05:21 2022] GET /identity/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:21.445376 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219bebc4-b78f-4582-9eab-d3bc92151e50 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:21.589558 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219bebc4-b78f-4582-9eab-d3bc92151e50 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:21.625875 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:21.634645 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6bf22e-a622-4d8c-b5f8-a2521430ab6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:21.635268 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6bf22e-a622-4d8c-b5f8-a2521430ab6f None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:21.635613 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6bf22e-a622-4d8c-b5f8-a2521430ab6f None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:21.635892 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6bf22e-a622-4d8c-b5f8-a2521430ab6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:21.636686 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6bf22e-a622-4d8c-b5f8-a2521430ab6f None admin] RBAC: Adding query filter params (role.id=e3235eda7c654e2abf96ed6a494f5152, scope.project.id=f9d86697e3eb4fd6b8f82a013cfcb5ba, user.id=bbcee328f8db4f5cb50f75830eb1cb5a) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:21.637050 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6bf22e-a622-4d8c-b5f8-a2521430ab6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:21.638575 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6bf22e-a622-4d8c-b5f8-a2521430ab6f None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:21.644996 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 74/147] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:05:21 2022] GET /identity/v3/role_assignments?user.id=bbcee328f8db4f5cb50f75830eb1cb5a&scope.project.id=f9d86697e3eb4fd6b8f82a013cfcb5ba&role.id=e3235eda7c654e2abf96ed6a494f5152 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:21.687790 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a5edcf-d7d8-4a3a-aeeb-ce7e844bfe0a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:21.835863 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a5edcf-d7d8-4a3a-aeeb-ce7e844bfe0a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:21.837540 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:21.883439 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c59e794-812c-4c6d-9c41-9f4aec1809c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:21.884095 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59e794-812c-4c6d-9c41-9f4aec1809c0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:21.884371 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59e794-812c-4c6d-9c41-9f4aec1809c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:21.884635 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59e794-812c-4c6d-9c41-9f4aec1809c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:21.885778 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c59e794-812c-4c6d-9c41-9f4aec1809c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:21.887327 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c59e794-812c-4c6d-9c41-9f4aec1809c0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:21.889209 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 74/148] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:05:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:21.894936 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6bf22e-a622-4d8c-b5f8-a2521430ab6f None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:22.038174 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6bf22e-a622-4d8c-b5f8-a2521430ab6f None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:22.074557 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e6de5-b012-4e94-9074-03a617f374f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:22.075220 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e6de5-b012-4e94-9074-03a617f374f6 None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:22.075500 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e6de5-b012-4e94-9074-03a617f374f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:22.075773 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e6de5-b012-4e94-9074-03a617f374f6 None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:22.076437 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11e6de5-b012-4e94-9074-03a617f374f6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:22.077971 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11e6de5-b012-4e94-9074-03a617f374f6 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:22.109091 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 75/149] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Aug 31 05:05:21 2022] POST /identity/v3/users => generated 283 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:05:22.260022 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d84c0-7b22-4eb4-bdbe-8cea5f1fc6aa None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:22.260323 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d84c0-7b22-4eb4-bdbe-8cea5f1fc6aa None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:22.260555 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d84c0-7b22-4eb4-bdbe-8cea5f1fc6aa None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:22.261909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 75/150] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:22.267774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a9c56-c5fd-4c23-b363-dbc6a79bdacf None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:22.268062 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a9c56-c5fd-4c23-b363-dbc6a79bdacf None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:22.268345 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a9c56-c5fd-4c23-b363-dbc6a79bdacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:22.294759 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-c23a9c56-c5fd-4c23-b363-dbc6a79bdacf None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:22.296018 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c23a9c56-c5fd-4c23-b363-dbc6a79bdacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:22.356412 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:22.382237 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 76/151] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:22.488821 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672b5ad-bb2b-4f4b-849a-1c5c428ad884 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:22.489164 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672b5ad-bb2b-4f4b-849a-1c5c428ad884 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:22.489358 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672b5ad-bb2b-4f4b-849a-1c5c428ad884 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:22.491005 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 76/152] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:22.497224 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9243cff3-3af2-45eb-a1c7-5221adc7199f None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:22.497508 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9243cff3-3af2-45eb-a1c7-5221adc7199f None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:22.497780 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9243cff3-3af2-45eb-a1c7-5221adc7199f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:22.521247 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-9243cff3-3af2-45eb-a1c7-5221adc7199f None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:22.522456 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-9243cff3-3af2-45eb-a1c7-5221adc7199f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:22.601587 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c7c9b9-681c-4cc0-af30-d5775bc46b3e None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:22.601924 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c7c9b9-681c-4cc0-af30-d5775bc46b3e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:22.602170 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c7c9b9-681c-4cc0-af30-d5775bc46b3e None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:22.603405 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 77/153] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:22.603525 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 77/154] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:22.609019 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a581301-f9ff-478b-9f56-96a07745c522 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:22.609311 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a581301-f9ff-478b-9f56-96a07745c522 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:22.609635 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a581301-f9ff-478b-9f56-96a07745c522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:22.628523 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-9a581301-f9ff-478b-9f56-96a07745c522 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:22.629712 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-9a581301-f9ff-478b-9f56-96a07745c522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:22.681645 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c7c9b9-681c-4cc0-af30-d5775bc46b3e None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:22.716461 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 78/155] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:22.815757 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c7c9b9-681c-4cc0-af30-d5775bc46b3e None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:22.851336 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9f4c9-e698-4d2d-baaf-ce17c88d0f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:22.851967 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9f4c9-e698-4d2d-baaf-ce17c88d0f43 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:22.852242 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9f4c9-e698-4d2d-baaf-ce17c88d0f43 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:22.852514 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9f4c9-e698-4d2d-baaf-ce17c88d0f43 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:22.863274 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b9f4c9-e698-4d2d-baaf-ce17c88d0f43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:22.868547 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-e9b9f4c9-e698-4d2d-baaf-ce17c88d0f43 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:05:22.868547 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:22.868547 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:22.868547 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:22.868547 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:22.868547 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:22.868547 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:22.868547 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:22.868547 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:22.868547 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:22.868547 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:22.868547 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:22.868547 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:22.868547 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:22.868547 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:22.868547 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:22.869411 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:22.869411 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:22.869411 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:22.869411 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:22.869411 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:22.869411 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:22.869411 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:22.869411 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:22.869411 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:22.869411 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:22.869411 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:22.869411 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:22.869411 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:22.869411 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:22.869411 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:22.869411 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:22.870250 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:22.870250 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:22.870250 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:22.870250 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:22.870250 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:22.870250 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:22.870250 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:22.870250 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:22.870250 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:22.870250 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:22.870250 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:05:22.870250 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:05:22.870250 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 78/156] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 05:05:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:05:22.876161 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a581301-f9ff-478b-9f56-96a07745c522 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:22.896535 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd53d69-6bfc-4fc0-b43a-05ec285dd24a None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:22.896867 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd53d69-6bfc-4fc0-b43a-05ec285dd24a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:22.897235 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd53d69-6bfc-4fc0-b43a-05ec285dd24a None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:22.899847 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 79/157] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 05:05:22.906110 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1810b79-943f-4ae4-a5b1-7d60d51bb446 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:22.906662 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1810b79-943f-4ae4-a5b1-7d60d51bb446 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:22.906859 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1810b79-943f-4ae4-a5b1-7d60d51bb446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:22.926612 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a1810b79-943f-4ae4-a5b1-7d60d51bb446 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:22.928004 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a1810b79-943f-4ae4-a5b1-7d60d51bb446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:22.930540 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:22.956458 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:23.013283 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a581301-f9ff-478b-9f56-96a07745c522 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:23.032534 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 80/158] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 05:05:23.033708 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1810b79-943f-4ae4-a5b1-7d60d51bb446 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:23.068599 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f872acd-710b-4ff3-babe-fd0b186e3549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:23.069565 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f872acd-710b-4ff3-babe-fd0b186e3549 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:23.070213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f872acd-710b-4ff3-babe-fd0b186e3549 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:23.071463 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f872acd-710b-4ff3-babe-fd0b186e3549 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:23.073098 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f872acd-710b-4ff3-babe-fd0b186e3549 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:23.073606 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f872acd-710b-4ff3-babe-fd0b186e3549 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:23.075352 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f872acd-710b-4ff3-babe-fd0b186e3549 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:23.087909 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 79/159] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:05:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:23.089103 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f872acd-710b-4ff3-babe-fd0b186e3549 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:23.161800 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1810b79-943f-4ae4-a5b1-7d60d51bb446 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:23.197601 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d91da2c-0cf6-4715-ad68-db6a49f41090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:23.198545 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d91da2c-0cf6-4715-ad68-db6a49f41090 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:23.198975 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d91da2c-0cf6-4715-ad68-db6a49f41090 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:23.199376 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d91da2c-0cf6-4715-ad68-db6a49f41090 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:23.212845 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d91da2c-0cf6-4715-ad68-db6a49f41090 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:23.213592 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f872acd-710b-4ff3-babe-fd0b186e3549 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:23.215169 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:23.219203 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-4d91da2c-0cf6-4715-ad68-db6a49f41090 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 05:05:23.219203 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:23.219203 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:23.219203 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:23.219203 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:23.219203 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:23.219203 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:23.219203 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:23.219203 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:23.219203 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:23.219203 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:23.219203 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:23.219203 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:23.219203 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:23.219203 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:23.219203 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:23.220378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:23.220378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:23.220378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:23.220378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:23.220378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:23.220378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:23.220378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:23.220378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:23.220378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:23.220378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:23.220378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:23.220378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:23.220378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:23.220378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:23.220378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:23.220378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:23.221382 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:23.221382 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:23.221382 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:23.221382 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:23.221382 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:23.221382 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:23.221382 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:23.221382 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:23.221382 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:23.221382 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:23.221382 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 05:05:23.221382 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:05:23.226135 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 81/160] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 05:05:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:05:23.226135 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d91da2c-0cf6-4715-ad68-db6a49f41090 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:23.255180 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:23.262163 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069fb386-6d2c-44b0-b010-a55937616ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:23.263442 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069fb386-6d2c-44b0-b010-a55937616ef4 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:23.264075 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069fb386-6d2c-44b0-b010-a55937616ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:23.264553 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069fb386-6d2c-44b0-b010-a55937616ef4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:23.277865 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069fb386-6d2c-44b0-b010-a55937616ef4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:23.285063 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-069fb386-6d2c-44b0-b010-a55937616ef4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 05:05:23.285063 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:23.285063 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:23.285063 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:23.285063 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:23.285063 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:23.285063 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:23.285063 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:23.285063 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:23.285063 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:23.285063 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:23.285063 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:23.285063 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:23.285063 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:23.285063 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:23.285063 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:23.286250 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:23.286250 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:23.286250 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:23.286250 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:23.286250 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:23.286250 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:23.286250 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:23.286250 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:23.286250 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:23.286250 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:23.286250 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:23.286250 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:23.286250 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:23.286250 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:23.286250 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:23.286250 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:23.287205 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:23.287205 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:23.287205 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:23.287205 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:23.287205 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:23.287205 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:23.287205 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:23.287205 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:23.287205 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:23.287205 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:23.287205 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 05:05:23.287205 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:05:23.289700 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 80/161] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:05:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:05:23.291038 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069fb386-6d2c-44b0-b010-a55937616ef4 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:23.351025 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d91da2c-0cf6-4715-ad68-db6a49f41090 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:23.387579 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b9abda-7386-451b-8f11-0466a278c6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:23.388577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9abda-7386-451b-8f11-0466a278c6b0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:23.389138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9abda-7386-451b-8f11-0466a278c6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:23.389615 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9abda-7386-451b-8f11-0466a278c6b0 None admin] PATH_INFO: `/v3/users/c96b7a6c0bc940caab4581734c1919ee` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:23.391054 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b9abda-7386-451b-8f11-0466a278c6b0 None admin] RBAC: Authorizing `identity:get_user(user_id=c96b7a6c0bc940caab4581734c1919ee)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:23.392720 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b9abda-7386-451b-8f11-0466a278c6b0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:23.394869 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 82/162] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:05:23 2022] GET /identity/v3/users/c96b7a6c0bc940caab4581734c1919ee => generated 247 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:23.395901 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b9abda-7386-451b-8f11-0466a278c6b0 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:23.397822 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069fb386-6d2c-44b0-b010-a55937616ef4 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:23.431863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e42f3a5-7eae-48bd-923d-4781837fd44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:23.432495 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42f3a5-7eae-48bd-923d-4781837fd44c None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:23.432685 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42f3a5-7eae-48bd-923d-4781837fd44c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:23.432930 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42f3a5-7eae-48bd-923d-4781837fd44c None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:23.433771 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e42f3a5-7eae-48bd-923d-4781837fd44c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:23.434350 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e42f3a5-7eae-48bd-923d-4781837fd44c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:23.435489 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e42f3a5-7eae-48bd-923d-4781837fd44c None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:23.446084 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 81/163] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:05:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:23.446084 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e42f3a5-7eae-48bd-923d-4781837fd44c None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:23.474414 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:23.532799 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:23.551469 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b9abda-7386-451b-8f11-0466a278c6b0 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:23.552724 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e42f3a5-7eae-48bd-923d-4781837fd44c None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:23.588762 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f9dd0c-653d-4577-ae56-ad370cfd5846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:23.589810 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f9dd0c-653d-4577-ae56-ad370cfd5846 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:23.590309 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f9dd0c-653d-4577-ae56-ad370cfd5846 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:23.590785 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f9dd0c-653d-4577-ae56-ad370cfd5846 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:23.619621 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c183bd39-c514-445f-b225-24b825387cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:23.620163 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183bd39-c514-445f-b225-24b825387cf8 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:23.620442 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183bd39-c514-445f-b225-24b825387cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:23.620709 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183bd39-c514-445f-b225-24b825387cf8 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:23.621384 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c183bd39-c514-445f-b225-24b825387cf8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:23.621681 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c183bd39-c514-445f-b225-24b825387cf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:23.623301 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c183bd39-c514-445f-b225-24b825387cf8 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:23.645843 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 83/164] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 05:05:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:23.646794 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c183bd39-c514-445f-b225-24b825387cf8 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:23.666087 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f9dd0c-653d-4577-ae56-ad370cfd5846 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:23.668003 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f9dd0c-653d-4577-ae56-ad370cfd5846 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:23.720349 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:23.798908 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c183bd39-c514-445f-b225-24b825387cf8 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:23.863713 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 82/165] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:05:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 05:05:23.864579 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f9dd0c-653d-4577-ae56-ad370cfd5846 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:23.869992 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61c395-f4b4-4c52-bcd0-8587fb2cbae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:23.870617 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61c395-f4b4-4c52-bcd0-8587fb2cbae1 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:23.870895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61c395-f4b4-4c52-bcd0-8587fb2cbae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:23.871161 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61c395-f4b4-4c52-bcd0-8587fb2cbae1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:23.893117 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa61c395-f4b4-4c52-bcd0-8587fb2cbae1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:23.899514 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-fa61c395-f4b4-4c52-bcd0-8587fb2cbae1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 05:05:23.899514 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:23.899514 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:23.899514 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:23.899514 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:23.899514 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:23.899514 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:23.899514 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:23.899514 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:23.899514 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:23.899514 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:23.899514 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:23.899514 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:23.899514 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:23.899514 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:23.899514 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:23.900383 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:23.900383 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:23.900383 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:23.900383 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:23.900383 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:23.900383 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:23.900383 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:23.900383 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:23.900383 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:23.900383 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:23.900383 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:23.900383 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:23.900383 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:23.900383 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:23.900383 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:23.900383 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:23.901167 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:23.901167 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:23.901167 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:23.901167 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:23.901167 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:23.901167 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:23.901167 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:23.901167 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:23.901167 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:23.901167 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:23.901167 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 05:05:23.901167 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:05:23.901167 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 84/166] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:05:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:05:23.910163 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61c395-f4b4-4c52-bcd0-8587fb2cbae1 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:23.937850 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:23.970203 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f9dd0c-653d-4577-ae56-ad370cfd5846 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:23.995201 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:24.006652 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d6f46-13b5-4772-b018-a55d17be99b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:24.007284 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d6f46-13b5-4772-b018-a55d17be99b0 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:24.007558 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d6f46-13b5-4772-b018-a55d17be99b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:24.007824 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d6f46-13b5-4772-b018-a55d17be99b0 None admin] PATH_INFO: `/v3/users/bbcee328f8db4f5cb50f75830eb1cb5a` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:24.008963 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625d6f46-13b5-4772-b018-a55d17be99b0 None admin] RBAC: Authorizing `identity:get_user(user_id=bbcee328f8db4f5cb50f75830eb1cb5a)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:24.010491 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625d6f46-13b5-4772-b018-a55d17be99b0 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:24.012411 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 83/167] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:05:23 2022] GET /identity/v3/users/bbcee328f8db4f5cb50f75830eb1cb5a => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:24.013210 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d6f46-13b5-4772-b018-a55d17be99b0 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:24.116057 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d6f46-13b5-4772-b018-a55d17be99b0 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:24.134689 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61c395-f4b4-4c52-bcd0-8587fb2cbae1 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:24.168137 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65ae67b-223c-4f5f-bde0-fe462d64c66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:24.168771 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ae67b-223c-4f5f-bde0-fe462d64c66c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:24.169052 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ae67b-223c-4f5f-bde0-fe462d64c66c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:24.169324 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ae67b-223c-4f5f-bde0-fe462d64c66c None admin] PATH_INFO: `/v3/users/bbcee328f8db4f5cb50f75830eb1cb5a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:24.170487 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65ae67b-223c-4f5f-bde0-fe462d64c66c None admin] RBAC: Authorizing `identity:get_user(user_id=bbcee328f8db4f5cb50f75830eb1cb5a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:24.172013 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65ae67b-223c-4f5f-bde0-fe462d64c66c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:24.173801 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9276d5b-929a-4cd0-a0ad-70389b83d813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:24.174059 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 85/168] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:05:23 2022] GET /identity/v3/users/bbcee328f8db4f5cb50f75830eb1cb5a => generated 275 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:24.174475 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9276d5b-929a-4cd0-a0ad-70389b83d813 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:24.174751 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9276d5b-929a-4cd0-a0ad-70389b83d813 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:24.174852 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65ae67b-223c-4f5f-bde0-fe462d64c66c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:24.175093 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9276d5b-929a-4cd0-a0ad-70389b83d813 None admin] PATH_INFO: `/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:24.177656 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9276d5b-929a-4cd0-a0ad-70389b83d813 None admin] RBAC: Authorizing `identity:get_project(project_id=f9d86697e3eb4fd6b8f82a013cfcb5ba)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:24.179177 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9276d5b-929a-4cd0-a0ad-70389b83d813 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:24.180814 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 84/169] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:05:24 2022] GET /identity/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:24.181568 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9276d5b-929a-4cd0-a0ad-70389b83d813 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:24.258113 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:24.295958 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65ae67b-223c-4f5f-bde0-fe462d64c66c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:24.322428 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:24.330544 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9276d5b-929a-4cd0-a0ad-70389b83d813 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:24.359891 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1035a43d-9d59-4e08-bc41-cc6421515f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:24.360875 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035a43d-9d59-4e08-bc41-cc6421515f73 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:24.361348 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035a43d-9d59-4e08-bc41-cc6421515f73 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:24.361807 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035a43d-9d59-4e08-bc41-cc6421515f73 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:24.362777 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1035a43d-9d59-4e08-bc41-cc6421515f73 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:24.363258 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1035a43d-9d59-4e08-bc41-cc6421515f73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:24.365193 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1035a43d-9d59-4e08-bc41-cc6421515f73 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:24.380434 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 86/170] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 05:05:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:24.381794 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1035a43d-9d59-4e08-bc41-cc6421515f73 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:24.399664 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b193e9c-210e-4389-8bae-ca8cfefb8432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:24.401114 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b193e9c-210e-4389-8bae-ca8cfefb8432 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:24.402284 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b193e9c-210e-4389-8bae-ca8cfefb8432 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:24.402854 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b193e9c-210e-4389-8bae-ca8cfefb8432 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:24.537436 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b193e9c-210e-4389-8bae-ca8cfefb8432 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:24.538946 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b193e9c-210e-4389-8bae-ca8cfefb8432 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:24.543656 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1035a43d-9d59-4e08-bc41-cc6421515f73 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:24.587708 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c6bbd3-1fa7-4e5c-a398-19437adca75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:24.588690 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6bbd3-1fa7-4e5c-a398-19437adca75e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:24.590298 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6bbd3-1fa7-4e5c-a398-19437adca75e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:24.590821 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6bbd3-1fa7-4e5c-a398-19437adca75e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:24.614049 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:24.650051 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:24.699701 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 85/171] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:05:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 05:05:24.700365 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b193e9c-210e-4389-8bae-ca8cfefb8432 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:24.719266 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c6bbd3-1fa7-4e5c-a398-19437adca75e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:24.720772 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c6bbd3-1fa7-4e5c-a398-19437adca75e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:24.856723 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 87/172] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:05:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 05:05:24.857583 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c6bbd3-1fa7-4e5c-a398-19437adca75e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:24.878786 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b193e9c-210e-4389-8bae-ca8cfefb8432 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:24.915428 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442379d-aa2a-4e5e-a168-faf9eeaa4ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:24.915964 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442379d-aa2a-4e5e-a168-faf9eeaa4ce9 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:24.916238 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442379d-aa2a-4e5e-a168-faf9eeaa4ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:24.916508 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442379d-aa2a-4e5e-a168-faf9eeaa4ce9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:24.917237 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442379d-aa2a-4e5e-a168-faf9eeaa4ce9 None admin] RBAC: Adding query filter params (role.id=c3d5589dd2864be9a0f26f5802d874cd, scope.project.id=f9d86697e3eb4fd6b8f82a013cfcb5ba, user.id=c96b7a6c0bc940caab4581734c1919ee) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:24.917542 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442379d-aa2a-4e5e-a168-faf9eeaa4ce9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:24.919027 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442379d-aa2a-4e5e-a168-faf9eeaa4ce9 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:24.922348 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:24.925310 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 86/173] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:05:24 2022] GET /identity/v3/role_assignments?user.id=c96b7a6c0bc940caab4581734c1919ee&scope.project.id=f9d86697e3eb4fd6b8f82a013cfcb5ba&role.id=c3d5589dd2864be9a0f26f5802d874cd => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:24.926152 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442379d-aa2a-4e5e-a168-faf9eeaa4ce9 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:24.953286 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:25.042266 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c6bbd3-1fa7-4e5c-a398-19437adca75e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:25.050807 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442379d-aa2a-4e5e-a168-faf9eeaa4ce9 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:25.102459 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31a494-2065-46fc-b234-db0ece29888c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:25.103122 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31a494-2065-46fc-b234-db0ece29888c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:25.103599 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31a494-2065-46fc-b234-db0ece29888c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:25.103881 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31a494-2065-46fc-b234-db0ece29888c None admin] PATH_INFO: `/v3/users/02e7eb819c754f4b9c880d300393dd5c` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:25.105188 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e31a494-2065-46fc-b234-db0ece29888c None admin] RBAC: Authorizing `identity:get_user(user_id=02e7eb819c754f4b9c880d300393dd5c)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:25.106898 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e31a494-2065-46fc-b234-db0ece29888c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:25.109024 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 88/174] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:05:24 2022] GET /identity/v3/users/02e7eb819c754f4b9c880d300393dd5c => generated 289 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:25.109852 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31a494-2065-46fc-b234-db0ece29888c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:25.137375 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbf1a4d-8a09-4c12-ae17-8f09597f6cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:25.138047 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf1a4d-8a09-4c12-ae17-8f09597f6cc1 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:25.138332 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf1a4d-8a09-4c12-ae17-8f09597f6cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:25.138605 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf1a4d-8a09-4c12-ae17-8f09597f6cc1 None admin] PATH_INFO: `/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:25.140667 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbf1a4d-8a09-4c12-ae17-8f09597f6cc1 None admin] RBAC: Authorizing `identity:get_project(project_id=f9d86697e3eb4fd6b8f82a013cfcb5ba)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:25.142195 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbf1a4d-8a09-4c12-ae17-8f09597f6cc1 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:25.143883 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 87/175] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:05:24 2022] GET /identity/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:25.145050 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2decb642-9682-413b-a0b8-1c6c86a8acb9 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:25.145320 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2decb642-9682-413b-a0b8-1c6c86a8acb9 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:25.145589 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2decb642-9682-413b-a0b8-1c6c86a8acb9 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:25.151101 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 88/176] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:25.151664 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2decb642-9682-413b-a0b8-1c6c86a8acb9 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:25.232983 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31a494-2065-46fc-b234-db0ece29888c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:25.245749 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:25.254467 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:25.264543 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2decb642-9682-413b-a0b8-1c6c86a8acb9 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:25.272185 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b2f22-4311-41c8-a4b1-5cf2a2c21d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:25.272836 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b2f22-4311-41c8-a4b1-5cf2a2c21d92 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:25.273115 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b2f22-4311-41c8-a4b1-5cf2a2c21d92 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:25.273385 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b2f22-4311-41c8-a4b1-5cf2a2c21d92 None admin] PATH_INFO: `/v3/projects/17a2a154edcf47af93f4f542d556d044` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:25.274597 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6b2f22-4311-41c8-a4b1-5cf2a2c21d92 None admin] RBAC: Authorizing `identity:get_project(project_id=17a2a154edcf47af93f4f542d556d044)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:25.276047 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6b2f22-4311-41c8-a4b1-5cf2a2c21d92 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:25.277855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 89/177] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:05:25 2022] GET /identity/v3/projects/17a2a154edcf47af93f4f542d556d044 => generated 312 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:25.279024 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777b50e-fbcc-46cd-b0e9-1f4bdf2677c0 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:25.279306 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777b50e-fbcc-46cd-b0e9-1f4bdf2677c0 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:25.279575 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777b50e-fbcc-46cd-b0e9-1f4bdf2677c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:25.299231 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-4777b50e-fbcc-46cd-b0e9-1f4bdf2677c0 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:25.300423 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4777b50e-fbcc-46cd-b0e9-1f4bdf2677c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:25.302595 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4237a-a8db-4c29-8393-8b53d868af6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:25.303238 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4237a-a8db-4c29-8393-8b53d868af6b None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:25.303536 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4237a-a8db-4c29-8393-8b53d868af6b None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:25.303793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4237a-a8db-4c29-8393-8b53d868af6b None admin] PATH_INFO: `/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:25.304925 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f4237a-a8db-4c29-8393-8b53d868af6b None admin] RBAC: Authorizing `identity:get_project(project_id=f9d86697e3eb4fd6b8f82a013cfcb5ba)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:25.306422 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f4237a-a8db-4c29-8393-8b53d868af6b None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:25.308068 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 89/178] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:05:25 2022] GET /identity/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:25.308823 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4237a-a8db-4c29-8393-8b53d868af6b None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:25.412753 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 90/179] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:25.413536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4777b50e-fbcc-46cd-b0e9-1f4bdf2677c0 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:25.432815 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4237a-a8db-4c29-8393-8b53d868af6b None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:25.475013 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1052eee-e9bf-4a85-934b-ee197b4ae80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:25.475668 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1052eee-e9bf-4a85-934b-ee197b4ae80f None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:25.475945 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1052eee-e9bf-4a85-934b-ee197b4ae80f None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:25.476222 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1052eee-e9bf-4a85-934b-ee197b4ae80f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:25.477098 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1052eee-e9bf-4a85-934b-ee197b4ae80f None admin] RBAC: Adding query filter params (role.id=a74bf01dd66f49f5988a197da269c822, scope.project.id=f9d86697e3eb4fd6b8f82a013cfcb5ba, user.id=bbcee328f8db4f5cb50f75830eb1cb5a) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:25.477705 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1052eee-e9bf-4a85-934b-ee197b4ae80f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:25.479208 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1052eee-e9bf-4a85-934b-ee197b4ae80f None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:25.486983 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 90/180] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:05:25 2022] GET /identity/v3/role_assignments?user.id=bbcee328f8db4f5cb50f75830eb1cb5a&scope.project.id=f9d86697e3eb4fd6b8f82a013cfcb5ba&role.id=a74bf01dd66f49f5988a197da269c822 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:25.487560 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1052eee-e9bf-4a85-934b-ee197b4ae80f None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:25.548524 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:25.580930 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4777b50e-fbcc-46cd-b0e9-1f4bdf2677c0 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:25.593452 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:25.668625 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a727d-5fe4-4477-8f07-35db8a169a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:25.669224 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a727d-5fe4-4477-8f07-35db8a169a6b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:25.669501 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a727d-5fe4-4477-8f07-35db8a169a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:25.669770 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a727d-5fe4-4477-8f07-35db8a169a6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:25.670604 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a727d-5fe4-4477-8f07-35db8a169a6b None admin] RBAC: Adding query filter params (role.id=e3235eda7c654e2abf96ed6a494f5152, scope.project.id=17a2a154edcf47af93f4f542d556d044, user.id=bbcee328f8db4f5cb50f75830eb1cb5a) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:25.671007 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a727d-5fe4-4477-8f07-35db8a169a6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:25.672633 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a727d-5fe4-4477-8f07-35db8a169a6b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:25.675411 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1052eee-e9bf-4a85-934b-ee197b4ae80f None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:25.684909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 91/181] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:05:25 2022] GET /identity/v3/role_assignments?user.id=bbcee328f8db4f5cb50f75830eb1cb5a&scope.project.id=17a2a154edcf47af93f4f542d556d044&role.id=e3235eda7c654e2abf96ed6a494f5152 => generated 608 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:25.685833 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c0b4c2-3a8d-4a21-b8fb-0c1b3b1c0f2f None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:25.686196 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c0b4c2-3a8d-4a21-b8fb-0c1b3b1c0f2f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:25.686463 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c0b4c2-3a8d-4a21-b8fb-0c1b3b1c0f2f None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:25.687846 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 92/182] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:25.689956 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a8c98-f203-4c12-9080-3a00b0137d20 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:25.689956 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a8c98-f203-4c12-9080-3a00b0137d20 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:25.689956 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a8c98-f203-4c12-9080-3a00b0137d20 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:25.692431 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 93/183] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:25.695864 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b28789-2f72-4e03-8791-7f0de1ffdd58 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:25.702464 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b28789-2f72-4e03-8791-7f0de1ffdd58 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:25.702860 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b28789-2f72-4e03-8791-7f0de1ffdd58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:25.726295 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d3b28789-2f72-4e03-8791-7f0de1ffdd58 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:25.727878 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d3b28789-2f72-4e03-8791-7f0de1ffdd58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:25.731761 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77abfa1a-9e73-4207-8e49-87358bd29b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:25.732456 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77abfa1a-9e73-4207-8e49-87358bd29b64 None admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:25.732738 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77abfa1a-9e73-4207-8e49-87358bd29b64 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:25.733012 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77abfa1a-9e73-4207-8e49-87358bd29b64 None admin] PATH_INFO: `/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba/users/02e7eb819c754f4b9c880d300393dd5c/roles/c87ef585cc814d919ef69ba258ecbbbd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:25.735827 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77abfa1a-9e73-4207-8e49-87358bd29b64 None admin] RBAC: Authorizing `identity:create_grant(project_id=f9d86697e3eb4fd6b8f82a013cfcb5ba, user_id=02e7eb819c754f4b9c880d300393dd5c, role_id=c87ef585cc814d919ef69ba258ecbbbd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:25.737445 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77abfa1a-9e73-4207-8e49-87358bd29b64 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:25.755707 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 91/184] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 05:05:25 2022] PUT /identity/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba/users/02e7eb819c754f4b9c880d300393dd5c/roles/c87ef585cc814d919ef69ba258ecbbbd => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:05:25.762268 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad3648-1b67-4b43-98d7-8ed9a5acac61 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:25.762268 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad3648-1b67-4b43-98d7-8ed9a5acac61 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:25.762268 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad3648-1b67-4b43-98d7-8ed9a5acac61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:25.783674 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-a8ad3648-1b67-4b43-98d7-8ed9a5acac61 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:25.785806 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a8ad3648-1b67-4b43-98d7-8ed9a5acac61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:25.822462 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 94/185] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:25.823460 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b28789-2f72-4e03-8791-7f0de1ffdd58 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:25.896571 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 92/186] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:25.941572 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:25.944930 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b28789-2f72-4e03-8791-7f0de1ffdd58 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:25.981005 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32d1d7-1f6a-4137-9116-25f16145ea41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:25.981635 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32d1d7-1f6a-4137-9116-25f16145ea41 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:25.981907 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32d1d7-1f6a-4137-9116-25f16145ea41 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:25.982201 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32d1d7-1f6a-4137-9116-25f16145ea41 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:25.992513 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e32d1d7-1f6a-4137-9116-25f16145ea41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:25.997624 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-2e32d1d7-1f6a-4137-9116-25f16145ea41 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:05:25.997624 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:25.997624 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:25.997624 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:25.997624 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:25.997624 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:25.997624 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:25.997624 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:25.997624 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:25.997624 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:25.997624 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:25.997624 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:25.997624 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:25.997624 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:25.997624 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:25.997624 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:25.998559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:25.998559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:25.998559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:25.998559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:25.998559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:25.998559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:25.998559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:25.998559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:25.998559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:25.998559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:25.998559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:25.998559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:25.998559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:25.998559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:25.998559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:25.998559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:25.999457 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:25.999457 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:25.999457 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:25.999457 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:25.999457 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:25.999457 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:25.999457 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:25.999457 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:25.999457 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:25.999457 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:25.999457 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:05:25.999457 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:05:25.999457 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 95/187] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 05:05:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:05:26.004840 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad3648-1b67-4b43-98d7-8ed9a5acac61 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:26.023897 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:26.119239 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad3648-1b67-4b43-98d7-8ed9a5acac61 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:26.130509 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32d1d7-1f6a-4137-9116-25f16145ea41 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:26.160127 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b46a73b-f7ec-4d16-b49b-d14f4cb3101c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:26.160127 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46a73b-f7ec-4d16-b49b-d14f4cb3101c None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:26.160426 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46a73b-f7ec-4d16-b49b-d14f4cb3101c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:26.160701 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46a73b-f7ec-4d16-b49b-d14f4cb3101c None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:26.161243 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b46a73b-f7ec-4d16-b49b-d14f4cb3101c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:26.161551 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b46a73b-f7ec-4d16-b49b-d14f4cb3101c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:26.163312 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b46a73b-f7ec-4d16-b49b-d14f4cb3101c None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:26.174959 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 93/188] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:05:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:26.180246 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b46a73b-f7ec-4d16-b49b-d14f4cb3101c None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:26.246398 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32d1d7-1f6a-4137-9116-25f16145ea41 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:26.260284 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:26.282486 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6a304-48d9-4d59-bee3-a33d445a60bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:26.283110 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6a304-48d9-4d59-bee3-a33d445a60bd None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:26.283389 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6a304-48d9-4d59-bee3-a33d445a60bd None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:26.283664 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6a304-48d9-4d59-bee3-a33d445a60bd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:26.285817 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b46a73b-f7ec-4d16-b49b-d14f4cb3101c None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:26.293798 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6a304-48d9-4d59-bee3-a33d445a60bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:26.298694 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-88a6a304-48d9-4d59-bee3-a33d445a60bd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 05:05:26.298694 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:26.298694 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:26.298694 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:26.298694 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:26.298694 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:26.298694 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:26.298694 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:26.298694 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:26.298694 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:26.298694 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:26.298694 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:26.298694 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:26.298694 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:26.298694 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:26.298694 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:26.299577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:26.299577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:26.299577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:26.299577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:26.299577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:26.299577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:26.299577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:26.299577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:26.299577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:26.299577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:26.299577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:26.299577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:26.299577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:26.299577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:26.299577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:26.299577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:26.300425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:26.300425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:26.300425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:26.300425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:26.300425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:26.300425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:26.300425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:26.300425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:26.300425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:26.300425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:26.300425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 05:05:26.300425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:05:26.300425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 96/189] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 05:05:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:05:26.301086 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6a304-48d9-4d59-bee3-a33d445a60bd None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:26.323394 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:26.326111 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62ae68-5577-4018-b781-58c8426e9548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:26.326954 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62ae68-5577-4018-b781-58c8426e9548 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:26.327284 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62ae68-5577-4018-b781-58c8426e9548 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:26.327541 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62ae68-5577-4018-b781-58c8426e9548 None admin] PATH_INFO: `/v3/users/eea7e5f8e74547a7bd6329f40a48dbc4` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:26.342023 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be62ae68-5577-4018-b781-58c8426e9548 None admin] RBAC: Authorizing `identity:get_user(user_id=eea7e5f8e74547a7bd6329f40a48dbc4)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:26.343567 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be62ae68-5577-4018-b781-58c8426e9548 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:26.345653 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 94/190] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:05:26 2022] GET /identity/v3/users/eea7e5f8e74547a7bd6329f40a48dbc4 => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:26.346452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62ae68-5577-4018-b781-58c8426e9548 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:26.407756 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6a304-48d9-4d59-bee3-a33d445a60bd None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:26.445146 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f3a654-a3f6-40cd-9069-c1f06d4d2ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:26.446059 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3a654-a3f6-40cd-9069-c1f06d4d2ca5 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:26.446340 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3a654-a3f6-40cd-9069-c1f06d4d2ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:26.446615 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3a654-a3f6-40cd-9069-c1f06d4d2ca5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:26.447722 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f3a654-a3f6-40cd-9069-c1f06d4d2ca5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:26.449182 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f3a654-a3f6-40cd-9069-c1f06d4d2ca5 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:26.451007 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 97/191] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:05:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:26.453333 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62ae68-5577-4018-b781-58c8426e9548 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:26.456326 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f3a654-a3f6-40cd-9069-c1f06d4d2ca5 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:26.491481 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5af4df4-ce51-4918-9226-09af331c4a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:26.492139 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af4df4-ce51-4918-9226-09af331c4a8a None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:26.492393 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af4df4-ce51-4918-9226-09af331c4a8a None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:26.492657 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af4df4-ce51-4918-9226-09af331c4a8a None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:26.493651 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5af4df4-ce51-4918-9226-09af331c4a8a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:26.493940 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5af4df4-ce51-4918-9226-09af331c4a8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:26.495417 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5af4df4-ce51-4918-9226-09af331c4a8a None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:26.496813 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:26.508480 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 95/192] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:05:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:26.508480 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5af4df4-ce51-4918-9226-09af331c4a8a None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:26.554923 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:26.568326 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f3a654-a3f6-40cd-9069-c1f06d4d2ca5 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:26.604335 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4823f4fd-6610-4cb1-b722-4c01fde166ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:26.604940 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823f4fd-6610-4cb1-b722-4c01fde166ee None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:26.605213 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823f4fd-6610-4cb1-b722-4c01fde166ee None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:26.605473 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823f4fd-6610-4cb1-b722-4c01fde166ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:26.610475 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5af4df4-ce51-4918-9226-09af331c4a8a None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:26.645385 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab44593-162d-4db6-bb32-a11fa060f7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:26.645993 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab44593-162d-4db6-bb32-a11fa060f7e8 None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:26.646378 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab44593-162d-4db6-bb32-a11fa060f7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:26.646637 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab44593-162d-4db6-bb32-a11fa060f7e8 None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:26.647186 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab44593-162d-4db6-bb32-a11fa060f7e8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:26.648667 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab44593-162d-4db6-bb32-a11fa060f7e8 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:26.678039 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 96/193] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Aug 31 05:05:26 2022] POST /identity/v3/users => generated 297 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:05:26.678039 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab44593-162d-4db6-bb32-a11fa060f7e8 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:26.704327 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4823f4fd-6610-4cb1-b722-4c01fde166ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:26.706478 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4823f4fd-6610-4cb1-b722-4c01fde166ee None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:26.739475 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:26.799956 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab44593-162d-4db6-bb32-a11fa060f7e8 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:26.825620 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:26.844422 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 98/194] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:05:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 05:05:26.846723 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b636d7f-d835-4a08-ac03-eb21bb284a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:26.846723 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b636d7f-d835-4a08-ac03-eb21bb284a08 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:26.846998 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b636d7f-d835-4a08-ac03-eb21bb284a08 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:26.847210 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b636d7f-d835-4a08-ac03-eb21bb284a08 None admin] PATH_INFO: `/v3/users/eea7e5f8e74547a7bd6329f40a48dbc4` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:26.848361 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b636d7f-d835-4a08-ac03-eb21bb284a08 None admin] RBAC: Authorizing `identity:get_user(user_id=eea7e5f8e74547a7bd6329f40a48dbc4)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:26.849829 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b636d7f-d835-4a08-ac03-eb21bb284a08 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:26.850512 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4823f4fd-6610-4cb1-b722-4c01fde166ee None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:26.851705 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 97/195] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:05:26 2022] GET /identity/v3/users/eea7e5f8e74547a7bd6329f40a48dbc4 => generated 283 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:26.857833 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b636d7f-d835-4a08-ac03-eb21bb284a08 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:26.976661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4823f4fd-6610-4cb1-b722-4c01fde166ee None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:26.979019 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b636d7f-d835-4a08-ac03-eb21bb284a08 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:27.002879 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:27.015036 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e9477-4cf3-4f85-9744-ae9f800ef752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:27.015684 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e9477-4cf3-4f85-9744-ae9f800ef752 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:27.015958 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e9477-4cf3-4f85-9744-ae9f800ef752 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:27.016271 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e9477-4cf3-4f85-9744-ae9f800ef752 None admin] PATH_INFO: `/v3/projects/b08ea5fe820543b6936ffde78e21ea4f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:27.017459 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69e9477-4cf3-4f85-9744-ae9f800ef752 None admin] RBAC: Authorizing `identity:get_project(project_id=b08ea5fe820543b6936ffde78e21ea4f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:27.018986 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69e9477-4cf3-4f85-9744-ae9f800ef752 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:27.020697 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 99/196] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:05:26 2022] GET /identity/v3/projects/b08ea5fe820543b6936ffde78e21ea4f => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:27.021907 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62b5b85-3a7b-430b-a8f8-289ab9a40cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:27.022731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b5b85-3a7b-430b-a8f8-289ab9a40cda None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:27.023150 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b5b85-3a7b-430b-a8f8-289ab9a40cda None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:27.023559 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b5b85-3a7b-430b-a8f8-289ab9a40cda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:27.026574 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e9477-4cf3-4f85-9744-ae9f800ef752 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:27.107456 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:27.120901 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62b5b85-3a7b-430b-a8f8-289ab9a40cda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:27.123364 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62b5b85-3a7b-430b-a8f8-289ab9a40cda None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:27.155257 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e9477-4cf3-4f85-9744-ae9f800ef752 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:27.204761 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf89dc-6eeb-41c3-a29e-291074a2ea4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:27.205380 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf89dc-6eeb-41c3-a29e-291074a2ea4d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:27.205683 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf89dc-6eeb-41c3-a29e-291074a2ea4d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:27.206560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf89dc-6eeb-41c3-a29e-291074a2ea4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:27.207311 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adf89dc-6eeb-41c3-a29e-291074a2ea4d None admin] RBAC: Adding query filter params (role.id=c3d5589dd2864be9a0f26f5802d874cd, scope.project.id=b08ea5fe820543b6936ffde78e21ea4f, user.id=eea7e5f8e74547a7bd6329f40a48dbc4) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:27.207888 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adf89dc-6eeb-41c3-a29e-291074a2ea4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:27.209638 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adf89dc-6eeb-41c3-a29e-291074a2ea4d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:27.218353 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 100/197] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:05:27 2022] GET /identity/v3/role_assignments?user.id=eea7e5f8e74547a7bd6329f40a48dbc4&scope.project.id=b08ea5fe820543b6936ffde78e21ea4f&role.id=c3d5589dd2864be9a0f26f5802d874cd => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:05:27.251950 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 98/198] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:05:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 05:05:27.258077 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf89dc-6eeb-41c3-a29e-291074a2ea4d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:27.391760 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf89dc-6eeb-41c3-a29e-291074a2ea4d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:27.415229 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:27.431223 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c1b12-97bb-406f-ab56-9396dff0ef15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:27.431864 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c1b12-97bb-406f-ab56-9396dff0ef15 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:27.432138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c1b12-97bb-406f-ab56-9396dff0ef15 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:27.432403 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c1b12-97bb-406f-ab56-9396dff0ef15 None admin] PATH_INFO: `/v3/projects/b08ea5fe820543b6936ffde78e21ea4f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:27.433549 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c1b12-97bb-406f-ab56-9396dff0ef15 None admin] RBAC: Authorizing `identity:get_project(project_id=b08ea5fe820543b6936ffde78e21ea4f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:27.435029 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c1b12-97bb-406f-ab56-9396dff0ef15 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:27.437057 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 101/199] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:05:27 2022] GET /identity/v3/projects/b08ea5fe820543b6936ffde78e21ea4f => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:27.442645 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62b5b85-3a7b-430b-a8f8-289ab9a40cda None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:27.465172 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:27.551172 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62b5b85-3a7b-430b-a8f8-289ab9a40cda None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:27.593126 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013a970-be15-4414-bc1e-c3e95eb39d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:27.593743 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013a970-be15-4414-bc1e-c3e95eb39d96 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:27.594034 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013a970-be15-4414-bc1e-c3e95eb39d96 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:27.594311 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013a970-be15-4414-bc1e-c3e95eb39d96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:27.595056 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e013a970-be15-4414-bc1e-c3e95eb39d96 None admin] RBAC: Adding query filter params (role.id=a74bf01dd66f49f5988a197da269c822, scope.project.id=b08ea5fe820543b6936ffde78e21ea4f, user.id=eea7e5f8e74547a7bd6329f40a48dbc4) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:27.595360 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e013a970-be15-4414-bc1e-c3e95eb39d96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:27.596840 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e013a970-be15-4414-bc1e-c3e95eb39d96 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:27.603104 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 99/200] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:05:27 2022] GET /identity/v3/role_assignments?user.id=eea7e5f8e74547a7bd6329f40a48dbc4&scope.project.id=b08ea5fe820543b6936ffde78e21ea4f&role.id=a74bf01dd66f49f5988a197da269c822 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:27.978864 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad4702-7633-4ef8-9f7c-e99ee9bea5db None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:27.979374 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad4702-7633-4ef8-9f7c-e99ee9bea5db None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:27.979374 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad4702-7633-4ef8-9f7c-e99ee9bea5db None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:27.980914 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 102/201] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:27.991327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db219dd0-21f9-4607-8c09-7a06ba63f718 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:27.991612 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db219dd0-21f9-4607-8c09-7a06ba63f718 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:27.991879 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db219dd0-21f9-4607-8c09-7a06ba63f718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:28.028835 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-db219dd0-21f9-4607-8c09-7a06ba63f718 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:28.030059 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-db219dd0-21f9-4607-8c09-7a06ba63f718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:28.111931 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 100/202] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 05:05:28.467046 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ad4702-7633-4ef8-9f7c-e99ee9bea5db None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:28.596770 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ad4702-7633-4ef8-9f7c-e99ee9bea5db None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:28.647584 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9692dacb-2890-4bd9-9d77-5062703091ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:28.648237 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692dacb-2890-4bd9-9d77-5062703091ed None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:28.648512 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692dacb-2890-4bd9-9d77-5062703091ed None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:28.648773 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692dacb-2890-4bd9-9d77-5062703091ed None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:28.665642 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9692dacb-2890-4bd9-9d77-5062703091ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:28.670555 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-9692dacb-2890-4bd9-9d77-5062703091ed None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 05:05:28.670555 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:28.670555 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:28.670555 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:28.670555 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:28.670555 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:28.670555 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:28.670555 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:28.670555 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:28.670555 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:28.670555 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:28.670555 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:28.670555 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:28.670555 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:28.670555 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:28.670555 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:28.671520 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:28.671520 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:28.671520 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:28.671520 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:28.671520 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:28.671520 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:28.671520 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:28.671520 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:28.671520 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:28.671520 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:28.671520 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:28.671520 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:28.671520 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:28.671520 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:28.671520 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:28.671520 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:28.672314 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:28.672314 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:28.672314 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:28.672314 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:28.672314 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:28.672314 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:28.672314 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:28.672314 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:28.672314 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:28.672314 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:28.672314 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 05:05:28.672314 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:05:28.672314 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 103/203] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:05:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:05:28.677966 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db219dd0-21f9-4607-8c09-7a06ba63f718 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:28.693544 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:28.820820 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db219dd0-21f9-4607-8c09-7a06ba63f718 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:28.877852 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf63037-e924-45ac-8420-e550eb9b32f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:28.878888 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf63037-e924-45ac-8420-e550eb9b32f9 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:28.879392 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf63037-e924-45ac-8420-e550eb9b32f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:28.879849 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf63037-e924-45ac-8420-e550eb9b32f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:28.880720 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf63037-e924-45ac-8420-e550eb9b32f9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:28.881226 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf63037-e924-45ac-8420-e550eb9b32f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:28.882838 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf63037-e924-45ac-8420-e550eb9b32f9 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:28.898224 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 101/204] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 05:05:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:28.903698 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9692dacb-2890-4bd9-9d77-5062703091ed None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:28.980468 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:29.033436 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9692dacb-2890-4bd9-9d77-5062703091ed None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:29.087683 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71be363f-4d2c-42bd-97d2-2d25c7e2be37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:29.088254 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be363f-4d2c-42bd-97d2-2d25c7e2be37 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:29.088532 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be363f-4d2c-42bd-97d2-2d25c7e2be37 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:29.088796 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be363f-4d2c-42bd-97d2-2d25c7e2be37 None admin] PATH_INFO: `/v3/users/02e7eb819c754f4b9c880d300393dd5c` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:29.089944 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71be363f-4d2c-42bd-97d2-2d25c7e2be37 None admin] RBAC: Authorizing `identity:get_user(user_id=02e7eb819c754f4b9c880d300393dd5c)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:29.091424 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71be363f-4d2c-42bd-97d2-2d25c7e2be37 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:29.093391 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 104/205] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:05:28 2022] GET /identity/v3/users/02e7eb819c754f4b9c880d300393dd5c => generated 289 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:29.098978 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf63037-e924-45ac-8420-e550eb9b32f9 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:29.208536 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:29.219626 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74f3af-f527-4934-b0c3-89011f4ce8cf None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:29.219917 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74f3af-f527-4934-b0c3-89011f4ce8cf None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:29.220170 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74f3af-f527-4934-b0c3-89011f4ce8cf None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:29.221660 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 105/206] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:29.226810 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b49a3-d8b3-435c-8287-30fea9cf6be5 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:29.227089 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b49a3-d8b3-435c-8287-30fea9cf6be5 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:29.227360 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b49a3-d8b3-435c-8287-30fea9cf6be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:29.254982 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-386b49a3-d8b3-435c-8287-30fea9cf6be5 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:29.256173 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-386b49a3-d8b3-435c-8287-30fea9cf6be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:29.264614 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf63037-e924-45ac-8420-e550eb9b32f9 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:29.328079 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f6a27d-e114-4c71-97ed-b96a46a09f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:29.328701 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6a27d-e114-4c71-97ed-b96a46a09f70 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:29.328979 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6a27d-e114-4c71-97ed-b96a46a09f70 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:29.329252 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6a27d-e114-4c71-97ed-b96a46a09f70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:29.375287 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 106/207] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 05:05:29.442383 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f6a27d-e114-4c71-97ed-b96a46a09f70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:29.443871 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f6a27d-e114-4c71-97ed-b96a46a09f70 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:29.487645 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e5bda-283c-44a2-9c46-7a747fd68a9a None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:29.488290 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e5bda-283c-44a2-9c46-7a747fd68a9a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:29.488832 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e5bda-283c-44a2-9c46-7a747fd68a9a None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:29.490514 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 107/208] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:29.495982 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590f8bc-741e-4599-be6c-df981d133601 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:29.496491 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590f8bc-741e-4599-be6c-df981d133601 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:29.496995 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590f8bc-741e-4599-be6c-df981d133601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:29.526435 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:29.532602 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-1590f8bc-741e-4599-be6c-df981d133601 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:29.534885 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1590f8bc-741e-4599-be6c-df981d133601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:29.557990 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 102/209] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:05:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 05:05:29.563785 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f6a27d-e114-4c71-97ed-b96a46a09f70 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:29.582724 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:29.608023 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 108/210] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:29.613708 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07803438-b23e-4378-8020-e9edfb005ae4 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:29.614742 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07803438-b23e-4378-8020-e9edfb005ae4 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:29.615241 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07803438-b23e-4378-8020-e9edfb005ae4 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:29.617498 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 109/211] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:29.627823 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dffda4-d5e9-4b90-a498-26ac765a4cef None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:29.628371 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dffda4-d5e9-4b90-a498-26ac765a4cef None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:29.628849 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dffda4-d5e9-4b90-a498-26ac765a4cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:29.662109 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e6dffda4-d5e9-4b90-a498-26ac765a4cef None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:29.663324 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e6dffda4-d5e9-4b90-a498-26ac765a4cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:29.709219 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f6a27d-e114-4c71-97ed-b96a46a09f70 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:29.748462 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e6bb4a-f8f7-43e2-8c3c-b49fcceb8515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:29.749139 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e6bb4a-f8f7-43e2-8c3c-b49fcceb8515 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:29.749454 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e6bb4a-f8f7-43e2-8c3c-b49fcceb8515 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:29.749725 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e6bb4a-f8f7-43e2-8c3c-b49fcceb8515 None admin] PATH_INFO: `/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:29.751636 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e6bb4a-f8f7-43e2-8c3c-b49fcceb8515 None admin] RBAC: Authorizing `identity:get_project(project_id=f9d86697e3eb4fd6b8f82a013cfcb5ba)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:29.753112 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e6bb4a-f8f7-43e2-8c3c-b49fcceb8515 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:29.755102 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 103/212] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:05:29 2022] GET /identity/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:05:29.756303 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e6bb4a-f8f7-43e2-8c3c-b49fcceb8515 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:29.783224 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 110/213] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 05:05:29.791083 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dffda4-d5e9-4b90-a498-26ac765a4cef None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:29.810939 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:29.868579 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e6bb4a-f8f7-43e2-8c3c-b49fcceb8515 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:29.905000 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14fab38-ffd3-40ad-bf7a-f458c667b113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:29.905581 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14fab38-ffd3-40ad-bf7a-f458c667b113 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:29.905857 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14fab38-ffd3-40ad-bf7a-f458c667b113 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:29.906158 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14fab38-ffd3-40ad-bf7a-f458c667b113 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:29.916849 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14fab38-ffd3-40ad-bf7a-f458c667b113 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:29.921731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-a14fab38-ffd3-40ad-bf7a-f458c667b113 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 05:05:29.921731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:29.921731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:29.921731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:29.921731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:29.921731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:29.921731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:29.921731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:29.921731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:29.921731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:29.921731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:29.921731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:29.921731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:29.921731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:29.921731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:29.921731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:29.922654 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:29.922654 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:29.922654 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:29.922654 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:29.922654 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:29.922654 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:29.922654 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:29.922654 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:29.922654 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:29.922654 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:29.922654 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:29.922654 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:29.922654 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:29.922654 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:29.922654 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:29.922654 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:29.923481 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:29.923481 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:29.923481 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:29.923481 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:29.923481 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:29.923481 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:29.923481 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:29.923481 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:29.923481 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:29.923481 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:29.923481 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 05:05:29.923481 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:05:29.923481 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 104/214] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:05:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:05:29.929886 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14fab38-ffd3-40ad-bf7a-f458c667b113 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:29.953879 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dffda4-d5e9-4b90-a498-26ac765a4cef None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:29.969335 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:29.989398 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecae7b1-f78d-468f-9846-92b2d39addb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:29.990322 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecae7b1-f78d-468f-9846-92b2d39addb6 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:29.990597 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecae7b1-f78d-468f-9846-92b2d39addb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:29.990858 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecae7b1-f78d-468f-9846-92b2d39addb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:29.992021 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecae7b1-f78d-468f-9846-92b2d39addb6 None admin] RBAC: Adding query filter params (role.id=c87ef585cc814d919ef69ba258ecbbbd, scope.project.id=f9d86697e3eb4fd6b8f82a013cfcb5ba, user.id=02e7eb819c754f4b9c880d300393dd5c) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:29.992342 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecae7b1-f78d-468f-9846-92b2d39addb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:29.994120 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecae7b1-f78d-468f-9846-92b2d39addb6 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:30.000074 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 111/215] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:05:29 2022] GET /identity/v3/role_assignments?user.id=02e7eb819c754f4b9c880d300393dd5c&scope.project.id=f9d86697e3eb4fd6b8f82a013cfcb5ba&role.id=c87ef585cc814d919ef69ba258ecbbbd => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:30.004984 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecae7b1-f78d-468f-9846-92b2d39addb6 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:30.050878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14fab38-ffd3-40ad-bf7a-f458c667b113 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:30.054364 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:30.101713 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c532159-3653-4a50-863d-a29d18d8d33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:30.102378 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c532159-3653-4a50-863d-a29d18d8d33d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:30.102656 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c532159-3653-4a50-863d-a29d18d8d33d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:30.102924 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c532159-3653-4a50-863d-a29d18d8d33d None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:30.103603 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c532159-3653-4a50-863d-a29d18d8d33d None admin] RBAC: Adding query filter params (name=member) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:30.103910 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c532159-3653-4a50-863d-a29d18d8d33d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:30.105888 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c532159-3653-4a50-863d-a29d18d8d33d None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:30.117350 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 105/216] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 05:05:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:30.121136 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba6d8b3-a147-4cad-81a4-10a6034d5671 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:30.121136 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba6d8b3-a147-4cad-81a4-10a6034d5671 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:30.121136 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba6d8b3-a147-4cad-81a4-10a6034d5671 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:30.122035 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 106/217] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:30.122768 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba6d8b3-a147-4cad-81a4-10a6034d5671 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:30.135968 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecae7b1-f78d-468f-9846-92b2d39addb6 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:30.175108 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e6f97-cb36-4e8d-ad99-2bc509a4425b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:30.175658 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e6f97-cb36-4e8d-ad99-2bc509a4425b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:30.175933 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e6f97-cb36-4e8d-ad99-2bc509a4425b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:30.176310 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e6f97-cb36-4e8d-ad99-2bc509a4425b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:30.177560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663e6f97-cb36-4e8d-ad99-2bc509a4425b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:30.179046 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663e6f97-cb36-4e8d-ad99-2bc509a4425b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:30.180845 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 112/218] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:05:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:30.181678 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e6f97-cb36-4e8d-ad99-2bc509a4425b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:30.233033 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:30.245422 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba6d8b3-a147-4cad-81a4-10a6034d5671 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:30.280382 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d318c06-faed-40fd-b0ab-05ccb689b1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:30.281055 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d318c06-faed-40fd-b0ab-05ccb689b1c0 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:30.281325 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d318c06-faed-40fd-b0ab-05ccb689b1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:30.281576 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d318c06-faed-40fd-b0ab-05ccb689b1c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:30.290753 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e6f97-cb36-4e8d-ad99-2bc509a4425b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:30.292239 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d318c06-faed-40fd-b0ab-05ccb689b1c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:30.297455 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-5d318c06-faed-40fd-b0ab-05ccb689b1c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:05:30.297455 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:30.297455 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:30.297455 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:30.297455 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:30.297455 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:30.297455 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:30.297455 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:30.297455 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:30.297455 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:30.297455 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:30.297455 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:30.297455 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:30.297455 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:30.297455 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:30.297455 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:30.298413 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:30.298413 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:30.298413 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:30.298413 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:30.298413 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:30.298413 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:30.298413 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:30.298413 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:30.298413 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:30.298413 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:30.298413 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:30.298413 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:30.298413 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:30.298413 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:30.298413 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:30.298413 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:30.304199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:30.304199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:30.304199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:30.304199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:30.304199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:30.304199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:30.304199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:30.304199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:30.304199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:30.304199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:30.304199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:05:30.304199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:05:30.304199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 107/219] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 05:05:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:05:30.304199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f73813-0a38-4874-a52a-d2cf1cfea128 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:30.304914 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f73813-0a38-4874-a52a-d2cf1cfea128 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:30.304914 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f73813-0a38-4874-a52a-d2cf1cfea128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:30.317156 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-65f73813-0a38-4874-a52a-d2cf1cfea128 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:30.318347 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-65f73813-0a38-4874-a52a-d2cf1cfea128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:30.323828 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b36bd-25b8-4264-bf9e-c8f3b0ebead2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:30.324490 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b36bd-25b8-4264-bf9e-c8f3b0ebead2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:30.324766 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b36bd-25b8-4264-bf9e-c8f3b0ebead2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:30.325031 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b36bd-25b8-4264-bf9e-c8f3b0ebead2 None admin] PATH_INFO: `/v3/users/de97b5d1390d4d4aa33ee9de7e090c79` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:30.337979 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:30.340368 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b36bd-25b8-4264-bf9e-c8f3b0ebead2 None admin] RBAC: Authorizing `identity:get_user(user_id=de97b5d1390d4d4aa33ee9de7e090c79)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:30.341821 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b36bd-25b8-4264-bf9e-c8f3b0ebead2 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:30.343631 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 113/220] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:05:30 2022] GET /identity/v3/users/de97b5d1390d4d4aa33ee9de7e090c79 => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:30.344369 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b36bd-25b8-4264-bf9e-c8f3b0ebead2 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:30.387937 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 108/221] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:30.388887 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f73813-0a38-4874-a52a-d2cf1cfea128 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:30.461255 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:30.469116 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b36bd-25b8-4264-bf9e-c8f3b0ebead2 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:30.495296 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f73813-0a38-4874-a52a-d2cf1cfea128 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:30.507810 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a764ce9-ecf8-45ed-b704-92c0354c69c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:30.508422 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a764ce9-ecf8-45ed-b704-92c0354c69c4 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:30.508698 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a764ce9-ecf8-45ed-b704-92c0354c69c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:30.508974 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a764ce9-ecf8-45ed-b704-92c0354c69c4 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:30.509616 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a764ce9-ecf8-45ed-b704-92c0354c69c4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:30.511115 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a764ce9-ecf8-45ed-b704-92c0354c69c4 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:30.531677 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91058062-524c-473d-9199-65a417f05248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:30.532321 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91058062-524c-473d-9199-65a417f05248 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:30.532751 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91058062-524c-473d-9199-65a417f05248 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:30.533046 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91058062-524c-473d-9199-65a417f05248 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:30.534059 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91058062-524c-473d-9199-65a417f05248 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:30.534381 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91058062-524c-473d-9199-65a417f05248 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:30.535894 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91058062-524c-473d-9199-65a417f05248 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:30.541374 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 114/222] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Aug 31 05:05:30 2022] POST /identity/v3/users => generated 297 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:05:30.550412 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a764ce9-ecf8-45ed-b704-92c0354c69c4 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:30.550523 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 109/223] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:05:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:30.551847 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91058062-524c-473d-9199-65a417f05248 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:30.576162 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:30.681329 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91058062-524c-473d-9199-65a417f05248 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:30.701387 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a764ce9-ecf8-45ed-b704-92c0354c69c4 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:30.718855 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae76a790-fdc4-49af-9d43-6db71a1df86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:30.719427 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76a790-fdc4-49af-9d43-6db71a1df86f None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:30.719703 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76a790-fdc4-49af-9d43-6db71a1df86f None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:30.719967 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76a790-fdc4-49af-9d43-6db71a1df86f None admin] PATH_INFO: `/v3/users/eea7e5f8e74547a7bd6329f40a48dbc4` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:30.721889 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae76a790-fdc4-49af-9d43-6db71a1df86f None admin] RBAC: Authorizing `identity:get_user(user_id=eea7e5f8e74547a7bd6329f40a48dbc4)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:30.723375 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae76a790-fdc4-49af-9d43-6db71a1df86f None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:30.724977 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 110/224] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:05:30 2022] GET /identity/v3/users/eea7e5f8e74547a7bd6329f40a48dbc4 => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:30.725744 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae76a790-fdc4-49af-9d43-6db71a1df86f None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:30.740829 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bad16d-7719-4d3f-91ee-950e4e757617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:30.740829 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bad16d-7719-4d3f-91ee-950e4e757617 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:30.741123 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bad16d-7719-4d3f-91ee-950e4e757617 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:30.741263 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bad16d-7719-4d3f-91ee-950e4e757617 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:30.769829 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:30.828088 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:30.837248 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bad16d-7719-4d3f-91ee-950e4e757617 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:30.839047 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bad16d-7719-4d3f-91ee-950e4e757617 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:30.848472 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae76a790-fdc4-49af-9d43-6db71a1df86f None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:30.890912 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a0db7-4400-4b17-99a3-608459c5af9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:30.891566 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a0db7-4400-4b17-99a3-608459c5af9a None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:30.891849 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a0db7-4400-4b17-99a3-608459c5af9a None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:30.892108 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a0db7-4400-4b17-99a3-608459c5af9a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:30.914896 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69a0db7-4400-4b17-99a3-608459c5af9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:30.923435 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-f69a0db7-4400-4b17-99a3-608459c5af9a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 05:05:30.923435 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:30.923435 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:30.923435 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:30.923435 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:30.923435 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:30.923435 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:30.923435 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:30.923435 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:30.923435 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:30.923435 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:30.923435 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:30.923435 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:30.923435 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:30.923435 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:30.923435 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:30.924346 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:30.924346 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:30.924346 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:30.924346 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:30.924346 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:30.924346 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:30.924346 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:30.924346 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:30.924346 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:30.924346 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:30.924346 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:30.924346 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:30.924346 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:30.924346 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:30.924346 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:30.924346 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:30.925141 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:30.925141 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:30.925141 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:30.925141 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:30.925141 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:30.925141 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:30.925141 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:30.925141 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:30.925141 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:30.925141 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:30.925141 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 05:05:30.925141 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:05:30.927380 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 111/225] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 05:05:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:05:30.928367 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a0db7-4400-4b17-99a3-608459c5af9a None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:30.970200 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 115/226] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:05:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 05:05:30.971162 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bad16d-7719-4d3f-91ee-950e4e757617 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:31.028291 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:31.067764 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a0db7-4400-4b17-99a3-608459c5af9a None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:31.087281 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bad16d-7719-4d3f-91ee-950e4e757617 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:31.123116 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c756bb4-44cc-4916-90aa-cc0bf61e77b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:31.123516 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bfe069-20cb-40fb-b9eb-e02da09c52a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:31.124183 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bfe069-20cb-40fb-b9eb-e02da09c52a1 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:31.124556 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bfe069-20cb-40fb-b9eb-e02da09c52a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:31.124819 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bfe069-20cb-40fb-b9eb-e02da09c52a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:31.125030 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c756bb4-44cc-4916-90aa-cc0bf61e77b1 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:31.125495 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c756bb4-44cc-4916-90aa-cc0bf61e77b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:31.125594 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bfe069-20cb-40fb-b9eb-e02da09c52a1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:31.125912 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bfe069-20cb-40fb-b9eb-e02da09c52a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:31.126292 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c756bb4-44cc-4916-90aa-cc0bf61e77b1 None admin] PATH_INFO: `/v3/projects/b08ea5fe820543b6936ffde78e21ea4f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:31.127352 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bfe069-20cb-40fb-b9eb-e02da09c52a1 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:31.127856 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c756bb4-44cc-4916-90aa-cc0bf61e77b1 None admin] RBAC: Authorizing `identity:get_project(project_id=b08ea5fe820543b6936ffde78e21ea4f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:31.130319 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c756bb4-44cc-4916-90aa-cc0bf61e77b1 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:31.132291 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 112/227] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:05:30 2022] GET /identity/v3/projects/b08ea5fe820543b6936ffde78e21ea4f => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:31.133811 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c756bb4-44cc-4916-90aa-cc0bf61e77b1 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:31.136128 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 116/228] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:05:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:31.141467 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bfe069-20cb-40fb-b9eb-e02da09c52a1 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:31.153974 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:31.248691 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bfe069-20cb-40fb-b9eb-e02da09c52a1 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:31.263499 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:31.274407 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c756bb4-44cc-4916-90aa-cc0bf61e77b1 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:31.286528 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c26092c-ed3e-4652-9f90-5a463736264b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:31.287235 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26092c-ed3e-4652-9f90-5a463736264b None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:31.287498 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26092c-ed3e-4652-9f90-5a463736264b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:31.287768 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26092c-ed3e-4652-9f90-5a463736264b None admin] PATH_INFO: `/v3/projects/b08ea5fe820543b6936ffde78e21ea4f/users/eea7e5f8e74547a7bd6329f40a48dbc4/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:31.289666 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c26092c-ed3e-4652-9f90-5a463736264b None admin] RBAC: Authorizing `identity:create_grant(project_id=b08ea5fe820543b6936ffde78e21ea4f, user_id=eea7e5f8e74547a7bd6329f40a48dbc4, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:31.291173 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c26092c-ed3e-4652-9f90-5a463736264b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:31.308053 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 117/229] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 05:05:31 2022] PUT /identity/v3/projects/b08ea5fe820543b6936ffde78e21ea4f/users/eea7e5f8e74547a7bd6329f40a48dbc4/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:05:31.309311 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c26092c-ed3e-4652-9f90-5a463736264b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:31.318586 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d7a825-e4e9-49d1-ba6f-ffefbe5f8f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:31.319608 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d7a825-e4e9-49d1-ba6f-ffefbe5f8f15 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:31.320082 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d7a825-e4e9-49d1-ba6f-ffefbe5f8f15 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:31.320596 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d7a825-e4e9-49d1-ba6f-ffefbe5f8f15 None admin] PATH_INFO: `/v3/projects/b08ea5fe820543b6936ffde78e21ea4f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:31.322083 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d7a825-e4e9-49d1-ba6f-ffefbe5f8f15 None admin] RBAC: Authorizing `identity:get_project(project_id=b08ea5fe820543b6936ffde78e21ea4f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:31.323891 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d7a825-e4e9-49d1-ba6f-ffefbe5f8f15 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:31.325909 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 113/230] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:05:31 2022] GET /identity/v3/projects/b08ea5fe820543b6936ffde78e21ea4f => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:31.335188 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d7a825-e4e9-49d1-ba6f-ffefbe5f8f15 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:31.495693 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:31.497812 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c26092c-ed3e-4652-9f90-5a463736264b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:31.517279 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d7a825-e4e9-49d1-ba6f-ffefbe5f8f15 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:31.535241 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4753405e-225e-4893-afab-d20aa8e30631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:31.536203 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753405e-225e-4893-afab-d20aa8e30631 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:31.536678 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753405e-225e-4893-afab-d20aa8e30631 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:31.537140 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753405e-225e-4893-afab-d20aa8e30631 None admin] PATH_INFO: `/v3/users/eea7e5f8e74547a7bd6329f40a48dbc4` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:31.538563 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4753405e-225e-4893-afab-d20aa8e30631 None admin] RBAC: Authorizing `identity:get_user(user_id=eea7e5f8e74547a7bd6329f40a48dbc4)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:31.540339 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4753405e-225e-4893-afab-d20aa8e30631 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:31.542292 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 118/231] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:05:31 2022] GET /identity/v3/users/eea7e5f8e74547a7bd6329f40a48dbc4 => generated 283 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:31.549282 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4753405e-225e-4893-afab-d20aa8e30631 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:31.556487 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123197c7-d4c3-4eee-a49a-5e5a703ded6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:31.557100 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123197c7-d4c3-4eee-a49a-5e5a703ded6f None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:31.557373 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123197c7-d4c3-4eee-a49a-5e5a703ded6f None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:31.557635 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123197c7-d4c3-4eee-a49a-5e5a703ded6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:31.558392 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123197c7-d4c3-4eee-a49a-5e5a703ded6f None admin] RBAC: Adding query filter params (role.id=e3235eda7c654e2abf96ed6a494f5152, scope.project.id=b08ea5fe820543b6936ffde78e21ea4f, user.id=de97b5d1390d4d4aa33ee9de7e090c79) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:31.558695 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123197c7-d4c3-4eee-a49a-5e5a703ded6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:31.560153 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123197c7-d4c3-4eee-a49a-5e5a703ded6f None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:31.566437 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 114/232] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:05:31 2022] GET /identity/v3/role_assignments?user.id=de97b5d1390d4d4aa33ee9de7e090c79&scope.project.id=b08ea5fe820543b6936ffde78e21ea4f&role.id=e3235eda7c654e2abf96ed6a494f5152 => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:31.639615 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:31.665929 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4753405e-225e-4893-afab-d20aa8e30631 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:31.702595 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731cc2b-b195-479e-8545-9133cc5a0bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:31.703226 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731cc2b-b195-479e-8545-9133cc5a0bdc None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:31.703497 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731cc2b-b195-479e-8545-9133cc5a0bdc None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:31.703811 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731cc2b-b195-479e-8545-9133cc5a0bdc None admin] PATH_INFO: `/v3/projects/b08ea5fe820543b6936ffde78e21ea4f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:31.704971 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e731cc2b-b195-479e-8545-9133cc5a0bdc None admin] RBAC: Authorizing `identity:get_project(project_id=b08ea5fe820543b6936ffde78e21ea4f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:31.706537 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e731cc2b-b195-479e-8545-9133cc5a0bdc None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:31.708370 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 119/233] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:05:31 2022] GET /identity/v3/projects/b08ea5fe820543b6936ffde78e21ea4f => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:31.714195 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123197c7-d4c3-4eee-a49a-5e5a703ded6f None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:31.822763 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123197c7-d4c3-4eee-a49a-5e5a703ded6f None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:31.856946 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88a678d-fc16-4c8f-b4df-ac3d2f004e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:31.857629 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a678d-fc16-4c8f-b4df-ac3d2f004e42 None admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:31.857900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a678d-fc16-4c8f-b4df-ac3d2f004e42 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:31.858190 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a678d-fc16-4c8f-b4df-ac3d2f004e42 None admin] PATH_INFO: `/v3/projects/b08ea5fe820543b6936ffde78e21ea4f/users/eea7e5f8e74547a7bd6329f40a48dbc4/roles/a74bf01dd66f49f5988a197da269c822` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:31.860413 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88a678d-fc16-4c8f-b4df-ac3d2f004e42 None admin] RBAC: Authorizing `identity:create_grant(project_id=b08ea5fe820543b6936ffde78e21ea4f, user_id=eea7e5f8e74547a7bd6329f40a48dbc4, role_id=a74bf01dd66f49f5988a197da269c822)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:31.861917 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88a678d-fc16-4c8f-b4df-ac3d2f004e42 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:31.886284 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 115/234] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 05:05:31 2022] PUT /identity/v3/projects/b08ea5fe820543b6936ffde78e21ea4f/users/eea7e5f8e74547a7bd6329f40a48dbc4/roles/a74bf01dd66f49f5988a197da269c822 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:05:32.779763 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c041446-f247-4eab-8adf-e3196ecf006e None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:32.780864 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c041446-f247-4eab-8adf-e3196ecf006e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:32.781342 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c041446-f247-4eab-8adf-e3196ecf006e None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:32.783218 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 120/235] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:32.788660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f30a5-2999-4582-98d3-8bf37f1e2015 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:32.789163 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f30a5-2999-4582-98d3-8bf37f1e2015 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:32.789622 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f30a5-2999-4582-98d3-8bf37f1e2015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:32.801367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8f779-fc71-49de-9ac2-defc16436c46 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:32.802234 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8f779-fc71-49de-9ac2-defc16436c46 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:32.802705 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8f779-fc71-49de-9ac2-defc16436c46 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:32.804115 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:32.804460 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 121/236] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:32.810059 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb44783-2505-4d78-a73b-ef69e3412c6f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:32.810346 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb44783-2505-4d78-a73b-ef69e3412c6f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:32.810652 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb44783-2505-4d78-a73b-ef69e3412c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:32.814481 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-307f30a5-2999-4582-98d3-8bf37f1e2015 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:32.815683 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-307f30a5-2999-4582-98d3-8bf37f1e2015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:32.842563 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-7cb44783-2505-4d78-a73b-ef69e3412c6f None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:32.844216 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7cb44783-2505-4d78-a73b-ef69e3412c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:32.940614 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 116/237] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:32.959434 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 122/238] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 05:05:33.257823 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307f30a5-2999-4582-98d3-8bf37f1e2015 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:33.328791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb44783-2505-4d78-a73b-ef69e3412c6f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:33.376406 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307f30a5-2999-4582-98d3-8bf37f1e2015 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:33.416986 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b76202-abef-4d49-8ea9-f73777c38fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:33.417977 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b76202-abef-4d49-8ea9-f73777c38fd7 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:33.418494 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b76202-abef-4d49-8ea9-f73777c38fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:33.418961 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b76202-abef-4d49-8ea9-f73777c38fd7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:33.420581 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:33.422080 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b76202-abef-4d49-8ea9-f73777c38fd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:33.423776 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b76202-abef-4d49-8ea9-f73777c38fd7 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:33.425566 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 117/239] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:05:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:33.431198 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b76202-abef-4d49-8ea9-f73777c38fd7 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:33.436003 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:33.562438 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb44783-2505-4d78-a73b-ef69e3412c6f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:33.591180 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b76202-abef-4d49-8ea9-f73777c38fd7 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:33.631541 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb57a8a8-0676-4833-b046-02076eaa1dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:33.632148 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57a8a8-0676-4833-b046-02076eaa1dc1 None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:33.632418 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57a8a8-0676-4833-b046-02076eaa1dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:33.634076 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57a8a8-0676-4833-b046-02076eaa1dc1 None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:33.634746 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb57a8a8-0676-4833-b046-02076eaa1dc1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:33.637850 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb57a8a8-0676-4833-b046-02076eaa1dc1 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:33.649985 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423cae16-4e3b-411f-955e-e093cab06d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:33.650654 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cae16-4e3b-411f-955e-e093cab06d0b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:33.650926 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cae16-4e3b-411f-955e-e093cab06d0b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:33.651191 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cae16-4e3b-411f-955e-e093cab06d0b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:33.666570 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423cae16-4e3b-411f-955e-e093cab06d0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:33.679137 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 118/240] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Aug 31 05:05:33 2022] POST /identity/v3/users => generated 293 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:05:33.680072 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-423cae16-4e3b-411f-955e-e093cab06d0b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 05:05:33.680072 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:33.680072 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:33.680072 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:33.680072 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:33.680072 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:33.680072 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:33.680072 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:33.680072 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:33.680072 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:33.680072 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:33.680072 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:33.680072 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:33.680072 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:33.680072 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:33.680072 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:33.680961 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:33.680961 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:33.680961 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:33.680961 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:33.680961 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:33.680961 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:33.680961 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:33.680961 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:33.680961 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:33.680961 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:33.680961 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:33.680961 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:33.680961 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:33.680961 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:33.680961 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:33.680961 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:33.681764 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:33.681764 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:33.681764 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:33.681764 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:33.681764 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:33.681764 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:33.681764 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:33.681764 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:33.681764 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:33.681764 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:33.681764 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 05:05:33.681764 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:05:33.681764 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 123/241] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:05:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:05:33.688130 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb57a8a8-0676-4833-b046-02076eaa1dc1 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:33.788422 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:33.837545 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c90b6c-65fb-44b1-b343-0dc01eda16ff None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:33.838217 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c90b6c-65fb-44b1-b343-0dc01eda16ff None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:33.838217 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c90b6c-65fb-44b1-b343-0dc01eda16ff None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:33.839710 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 124/242] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:33.845805 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee83bd7-f688-4091-9e71-9019ee791670 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:33.846603 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee83bd7-f688-4091-9e71-9019ee791670 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:33.847063 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee83bd7-f688-4091-9e71-9019ee791670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:33.848242 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb57a8a8-0676-4833-b046-02076eaa1dc1 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:33.870861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-cee83bd7-f688-4091-9e71-9019ee791670 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:33.872467 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-cee83bd7-f688-4091-9e71-9019ee791670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:33.891150 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c531c498-b172-4f98-a1b2-f044cbd6d48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:33.891722 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531c498-b172-4f98-a1b2-f044cbd6d48e None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:33.892002 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531c498-b172-4f98-a1b2-f044cbd6d48e None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:33.892273 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531c498-b172-4f98-a1b2-f044cbd6d48e None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:33.893067 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c531c498-b172-4f98-a1b2-f044cbd6d48e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:33.893373 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c531c498-b172-4f98-a1b2-f044cbd6d48e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:33.894909 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c531c498-b172-4f98-a1b2-f044cbd6d48e None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:33.904855 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 119/243] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 05:05:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:33.911337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c531c498-b172-4f98-a1b2-f044cbd6d48e None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:33.948572 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 125/244] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 05:05:33.994066 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b4baf-b03d-4c66-9aa5-f582c317adb7 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:33.994350 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b4baf-b03d-4c66-9aa5-f582c317adb7 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:33.994649 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b4baf-b03d-4c66-9aa5-f582c317adb7 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:33.998907 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 126/245] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:34.001116 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8529b4-3469-4b33-b781-1c18977b2577 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:34.001498 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8529b4-3469-4b33-b781-1c18977b2577 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:34.001863 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8529b4-3469-4b33-b781-1c18977b2577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:34.023188 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c531c498-b172-4f98-a1b2-f044cbd6d48e None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:34.027419 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-8c8529b4-3469-4b33-b781-1c18977b2577 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:34.028702 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8c8529b4-3469-4b33-b781-1c18977b2577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:34.051793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:34.062126 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:34.070366 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e3b8c3-ecc3-42e2-a755-dd8d7a2b407f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:34.071310 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3b8c3-ecc3-42e2-a755-dd8d7a2b407f None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:34.071808 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3b8c3-ecc3-42e2-a755-dd8d7a2b407f None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:34.072219 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3b8c3-ecc3-42e2-a755-dd8d7a2b407f None admin] PATH_INFO: `/v3/users/91a0b0b8f8174d7b80fe85ba36c32a11` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:34.090688 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e3b8c3-ecc3-42e2-a755-dd8d7a2b407f None admin] RBAC: Authorizing `identity:get_user(user_id=91a0b0b8f8174d7b80fe85ba36c32a11)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:34.092745 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e3b8c3-ecc3-42e2-a755-dd8d7a2b407f None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:34.095032 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 120/246] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:05:33 2022] GET /identity/v3/users/91a0b0b8f8174d7b80fe85ba36c32a11 => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:34.100740 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e3b8c3-ecc3-42e2-a755-dd8d7a2b407f None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:34.116221 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 127/247] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:34.219645 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e3b8c3-ecc3-42e2-a755-dd8d7a2b407f None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:34.259515 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2336c90b-e601-4bab-b487-cbd993486614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:34.260177 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2336c90b-e601-4bab-b487-cbd993486614 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:34.260750 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2336c90b-e601-4bab-b487-cbd993486614 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:34.260854 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2336c90b-e601-4bab-b487-cbd993486614 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:34.262096 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8529b4-3469-4b33-b781-1c18977b2577 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:34.346570 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:34.381777 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2336c90b-e601-4bab-b487-cbd993486614 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:34.383325 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2336c90b-e601-4bab-b487-cbd993486614 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:34.410335 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8529b4-3469-4b33-b781-1c18977b2577 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:34.452213 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e391554-96f0-4c8e-bada-5c813562eeb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:34.453358 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e391554-96f0-4c8e-bada-5c813562eeb5 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:34.453644 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e391554-96f0-4c8e-bada-5c813562eeb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:34.453920 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e391554-96f0-4c8e-bada-5c813562eeb5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:34.475019 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e391554-96f0-4c8e-bada-5c813562eeb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:34.482473 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-2e391554-96f0-4c8e-bada-5c813562eeb5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:05:34.482473 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:34.482473 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:34.482473 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:34.482473 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:34.482473 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:34.482473 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:34.482473 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:34.482473 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:34.482473 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:34.482473 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:34.482473 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:34.482473 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:34.482473 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:34.482473 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:34.482473 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:34.483375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:34.483375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:34.483375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:34.483375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:34.483375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:34.483375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:34.483375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:34.483375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:34.483375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:34.483375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:34.483375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:34.483375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:34.483375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:34.483375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:34.483375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:34.483375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:34.484264 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:34.484264 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:34.484264 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:34.484264 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:34.484264 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:34.484264 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:34.484264 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:34.484264 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:34.484264 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:34.484264 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:34.484264 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:05:34.484264 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:05:34.484264 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 128/248] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 05:05:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:05:34.485434 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe6de21-210f-46bf-af82-fdeb4ae0ad8a None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:34.486790 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe6de21-210f-46bf-af82-fdeb4ae0ad8a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:34.487074 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe6de21-210f-46bf-af82-fdeb4ae0ad8a None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:34.492360 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 129/249] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:34.492360 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe6de21-210f-46bf-af82-fdeb4ae0ad8a None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:34.506764 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 121/250] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:05:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 05:05:34.507906 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2336c90b-e601-4bab-b487-cbd993486614 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:34.530498 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:34.601643 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe6de21-210f-46bf-af82-fdeb4ae0ad8a None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:34.606237 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:34.629444 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2336c90b-e601-4bab-b487-cbd993486614 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:34.648054 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee54efb2-d7ae-40db-ace6-2ea29b4526fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:34.648934 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54efb2-d7ae-40db-ace6-2ea29b4526fc None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:34.649407 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54efb2-d7ae-40db-ace6-2ea29b4526fc None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:34.649869 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54efb2-d7ae-40db-ace6-2ea29b4526fc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:34.665716 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51532552-79eb-4c2f-9e39-2b28eddd2618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:34.666426 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51532552-79eb-4c2f-9e39-2b28eddd2618 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:34.666712 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51532552-79eb-4c2f-9e39-2b28eddd2618 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:34.666983 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51532552-79eb-4c2f-9e39-2b28eddd2618 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:34.667856 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51532552-79eb-4c2f-9e39-2b28eddd2618 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:34.668084 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51532552-79eb-4c2f-9e39-2b28eddd2618 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:34.669046 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee54efb2-d7ae-40db-ace6-2ea29b4526fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:34.669585 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51532552-79eb-4c2f-9e39-2b28eddd2618 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:34.674963 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-ee54efb2-d7ae-40db-ace6-2ea29b4526fc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 05:05:34.674963 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:34.674963 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:34.674963 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:34.674963 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:34.674963 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:34.674963 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:34.674963 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:34.674963 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:34.674963 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:34.674963 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:34.674963 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:34.674963 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:34.674963 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:34.674963 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:34.674963 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:34.675877 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:34.675877 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:34.675877 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:34.675877 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:34.675877 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:34.675877 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:34.675877 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:34.675877 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:34.675877 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:34.675877 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:34.675877 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:34.675877 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:34.675877 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:34.675877 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:34.675877 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:34.675877 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:34.676696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:34.676696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:34.676696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:34.676696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:34.676696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:34.676696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:34.676696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:34.676696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:34.676696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:34.676696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:34.676696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 05:05:34.676696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:05:34.679617 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 130/251] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:05:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:05:34.681156 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ecb963-4c7e-4903-a53b-b792c77a69b9 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:34.681566 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ecb963-4c7e-4903-a53b-b792c77a69b9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:34.681974 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ecb963-4c7e-4903-a53b-b792c77a69b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:34.682225 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 122/252] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:05:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:34.683135 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51532552-79eb-4c2f-9e39-2b28eddd2618 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:34.704450 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-c8ecb963-4c7e-4903-a53b-b792c77a69b9 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:34.705906 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c8ecb963-4c7e-4903-a53b-b792c77a69b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:34.794239 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51532552-79eb-4c2f-9e39-2b28eddd2618 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:34.794501 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 131/253] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:34.795585 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ecb963-4c7e-4903-a53b-b792c77a69b9 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:34.815952 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:34.839787 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd42915-61b1-464c-9189-0bfd3a152106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:34.840760 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd42915-61b1-464c-9189-0bfd3a152106 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:34.841099 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd42915-61b1-464c-9189-0bfd3a152106 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:34.841473 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd42915-61b1-464c-9189-0bfd3a152106 None admin] PATH_INFO: `/v3/projects/b08ea5fe820543b6936ffde78e21ea4f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:34.845659 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd42915-61b1-464c-9189-0bfd3a152106 None admin] RBAC: Authorizing `identity:get_project(project_id=b08ea5fe820543b6936ffde78e21ea4f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:34.847424 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd42915-61b1-464c-9189-0bfd3a152106 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:34.849438 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 123/254] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:05:34 2022] GET /identity/v3/projects/b08ea5fe820543b6936ffde78e21ea4f => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:34.850136 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd42915-61b1-464c-9189-0bfd3a152106 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:34.886498 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:34.914939 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ecb963-4c7e-4903-a53b-b792c77a69b9 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:34.949954 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b151a5-2cd0-4a29-96dc-7fb0cee3b36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:34.950939 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b151a5-2cd0-4a29-96dc-7fb0cee3b36d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:34.951414 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b151a5-2cd0-4a29-96dc-7fb0cee3b36d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:34.951898 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b151a5-2cd0-4a29-96dc-7fb0cee3b36d None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:34.952790 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b151a5-2cd0-4a29-96dc-7fb0cee3b36d None admin] RBAC: Adding query filter params (name=member) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:34.953297 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b151a5-2cd0-4a29-96dc-7fb0cee3b36d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:34.954913 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b151a5-2cd0-4a29-96dc-7fb0cee3b36d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:34.963924 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 132/255] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 05:05:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:34.964982 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b151a5-2cd0-4a29-96dc-7fb0cee3b36d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:34.969252 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd42915-61b1-464c-9189-0bfd3a152106 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:35.007086 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521c24e-cfb2-49dd-9f4b-4044616d20ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:35.007781 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521c24e-cfb2-49dd-9f4b-4044616d20ab None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:35.008061 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521c24e-cfb2-49dd-9f4b-4044616d20ab None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:35.008347 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521c24e-cfb2-49dd-9f4b-4044616d20ab None admin] PATH_INFO: `/v3/users/eea7e5f8e74547a7bd6329f40a48dbc4` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:35.009610 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f521c24e-cfb2-49dd-9f4b-4044616d20ab None admin] RBAC: Authorizing `identity:get_user(user_id=eea7e5f8e74547a7bd6329f40a48dbc4)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:35.011339 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f521c24e-cfb2-49dd-9f4b-4044616d20ab None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:35.013270 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 124/256] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:05:34 2022] GET /identity/v3/users/eea7e5f8e74547a7bd6329f40a48dbc4 => generated 283 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:35.014880 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521c24e-cfb2-49dd-9f4b-4044616d20ab None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:35.038741 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:35.091657 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b151a5-2cd0-4a29-96dc-7fb0cee3b36d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:35.102376 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:35.127216 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd55dee8-7d7b-4795-8c73-f3f89e0aa088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:35.128149 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521c24e-cfb2-49dd-9f4b-4044616d20ab None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:35.129055 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd55dee8-7d7b-4795-8c73-f3f89e0aa088 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:35.129344 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd55dee8-7d7b-4795-8c73-f3f89e0aa088 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:35.129669 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd55dee8-7d7b-4795-8c73-f3f89e0aa088 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:35.131043 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd55dee8-7d7b-4795-8c73-f3f89e0aa088 None admin] RBAC: Adding query filter params (role.id=c87ef585cc814d919ef69ba258ecbbbd, scope.project.id=b08ea5fe820543b6936ffde78e21ea4f, user.id=91a0b0b8f8174d7b80fe85ba36c32a11) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:35.131564 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd55dee8-7d7b-4795-8c73-f3f89e0aa088 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:35.133229 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd55dee8-7d7b-4795-8c73-f3f89e0aa088 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:35.139368 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 133/257] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:05:34 2022] GET /identity/v3/role_assignments?user.id=91a0b0b8f8174d7b80fe85ba36c32a11&scope.project.id=b08ea5fe820543b6936ffde78e21ea4f&role.id=c87ef585cc814d919ef69ba258ecbbbd => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:35.140205 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd55dee8-7d7b-4795-8c73-f3f89e0aa088 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:35.201530 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ddd801-1c77-449c-bf9d-bfbbee023bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:35.202240 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ddd801-1c77-449c-bf9d-bfbbee023bfa None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:35.202520 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ddd801-1c77-449c-bf9d-bfbbee023bfa None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:35.202795 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ddd801-1c77-449c-bf9d-bfbbee023bfa None admin] PATH_INFO: `/v3/users/de97b5d1390d4d4aa33ee9de7e090c79` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:35.204013 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ddd801-1c77-449c-bf9d-bfbbee023bfa None admin] RBAC: Authorizing `identity:get_user(user_id=de97b5d1390d4d4aa33ee9de7e090c79)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:35.205529 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ddd801-1c77-449c-bf9d-bfbbee023bfa None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:35.207632 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 125/258] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:05:35 2022] GET /identity/v3/users/de97b5d1390d4d4aa33ee9de7e090c79 => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:35.208462 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ddd801-1c77-449c-bf9d-bfbbee023bfa None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:35.316384 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd55dee8-7d7b-4795-8c73-f3f89e0aa088 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:35.373439 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ddd801-1c77-449c-bf9d-bfbbee023bfa None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:35.378552 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef478209-c2c3-44a9-b628-b83aaea76eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:35.379370 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef478209-c2c3-44a9-b628-b83aaea76eda None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:35.379890 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef478209-c2c3-44a9-b628-b83aaea76eda None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:35.380466 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef478209-c2c3-44a9-b628-b83aaea76eda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:35.388562 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:35.442571 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e71902-10f2-4f6d-b0b7-d107195c369c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:35.443526 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e71902-10f2-4f6d-b0b7-d107195c369c None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:35.444006 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e71902-10f2-4f6d-b0b7-d107195c369c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:35.444453 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e71902-10f2-4f6d-b0b7-d107195c369c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:35.466053 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e71902-10f2-4f6d-b0b7-d107195c369c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:35.471652 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-b4e71902-10f2-4f6d-b0b7-d107195c369c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 05:05:35.471652 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:35.471652 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:35.471652 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:35.471652 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:35.471652 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:35.471652 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:35.471652 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:35.471652 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:35.471652 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:35.471652 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:35.471652 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:35.471652 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:35.471652 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:35.471652 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:35.471652 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:35.472629 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:35.472629 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:35.472629 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:35.472629 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:35.472629 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:35.472629 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:35.472629 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:35.472629 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:35.472629 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:35.472629 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:35.472629 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:35.472629 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:35.472629 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:35.472629 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:35.472629 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:35.472629 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:35.473468 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:35.473468 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:35.473468 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:35.473468 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:35.473468 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:35.473468 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:35.473468 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:35.473468 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:35.473468 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:35.473468 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:35.473468 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 05:05:35.473468 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:05:35.475779 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 126/259] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 05:05:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:05:35.476931 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e71902-10f2-4f6d-b0b7-d107195c369c None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:35.490343 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:35.504443 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef478209-c2c3-44a9-b628-b83aaea76eda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:35.505945 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef478209-c2c3-44a9-b628-b83aaea76eda None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:35.612511 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 134/260] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:05:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 05:05:35.613429 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef478209-c2c3-44a9-b628-b83aaea76eda None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:35.648807 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:35.716764 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef478209-c2c3-44a9-b628-b83aaea76eda None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:35.753500 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e184f-3083-433e-a863-bc2d47153d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:35.754357 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e184f-3083-433e-a863-bc2d47153d37 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:35.754704 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e184f-3083-433e-a863-bc2d47153d37 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:35.755039 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e184f-3083-433e-a863-bc2d47153d37 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:35.755779 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34e184f-3083-433e-a863-bc2d47153d37 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:35.756144 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34e184f-3083-433e-a863-bc2d47153d37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:35.757603 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34e184f-3083-433e-a863-bc2d47153d37 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:35.767201 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 135/261] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:05:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:35.768200 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e184f-3083-433e-a863-bc2d47153d37 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:35.781764 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e71902-10f2-4f6d-b0b7-d107195c369c None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:35.831418 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346287ae-0584-43d1-863c-17f618ac15da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:35.832228 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346287ae-0584-43d1-863c-17f618ac15da None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:35.832512 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346287ae-0584-43d1-863c-17f618ac15da None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:35.832786 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346287ae-0584-43d1-863c-17f618ac15da None admin] PATH_INFO: `/v3/projects/b08ea5fe820543b6936ffde78e21ea4f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:35.833924 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346287ae-0584-43d1-863c-17f618ac15da None admin] RBAC: Authorizing `identity:get_project(project_id=b08ea5fe820543b6936ffde78e21ea4f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:35.838027 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346287ae-0584-43d1-863c-17f618ac15da None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:35.838027 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 127/262] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:05:35 2022] GET /identity/v3/projects/b08ea5fe820543b6936ffde78e21ea4f => generated 302 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:35.838325 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346287ae-0584-43d1-863c-17f618ac15da None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:35.928653 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:35.953043 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346287ae-0584-43d1-863c-17f618ac15da None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:35.989738 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9427351-afc2-486b-9f35-43626bf90506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:35.990527 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9427351-afc2-486b-9f35-43626bf90506 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:35.994036 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9427351-afc2-486b-9f35-43626bf90506 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:35.994036 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9427351-afc2-486b-9f35-43626bf90506 None admin] PATH_INFO: `/v3/users/eea7e5f8e74547a7bd6329f40a48dbc4` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:35.995765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9427351-afc2-486b-9f35-43626bf90506 None admin] RBAC: Authorizing `identity:get_user(user_id=eea7e5f8e74547a7bd6329f40a48dbc4)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:35.997233 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9427351-afc2-486b-9f35-43626bf90506 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:35.999083 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 128/263] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:05:35 2022] GET /identity/v3/users/eea7e5f8e74547a7bd6329f40a48dbc4 => generated 283 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:36.003848 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9427351-afc2-486b-9f35-43626bf90506 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:36.010630 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e184f-3083-433e-a863-bc2d47153d37 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:36.018304 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:36.048012 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b64617-c36c-4a8a-89b8-c2a148a9782b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:36.050546 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b64617-c36c-4a8a-89b8-c2a148a9782b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:36.050823 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b64617-c36c-4a8a-89b8-c2a148a9782b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:36.051095 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b64617-c36c-4a8a-89b8-c2a148a9782b None admin] PATH_INFO: `/v3/projects/b08ea5fe820543b6936ffde78e21ea4f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:36.053399 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b64617-c36c-4a8a-89b8-c2a148a9782b None admin] RBAC: Authorizing `identity:get_project(project_id=b08ea5fe820543b6936ffde78e21ea4f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:36.054877 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b64617-c36c-4a8a-89b8-c2a148a9782b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:36.056534 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 136/264] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:05:35 2022] GET /identity/v3/projects/b08ea5fe820543b6936ffde78e21ea4f => generated 302 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:36.057302 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b64617-c36c-4a8a-89b8-c2a148a9782b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:36.213203 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9427351-afc2-486b-9f35-43626bf90506 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:36.251733 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c1d548-52d2-4040-94a4-46ab274ff635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:36.252426 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1d548-52d2-4040-94a4-46ab274ff635 None admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:36.252703 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1d548-52d2-4040-94a4-46ab274ff635 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:36.252975 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1d548-52d2-4040-94a4-46ab274ff635 None admin] PATH_INFO: `/v3/projects/b08ea5fe820543b6936ffde78e21ea4f/users/de97b5d1390d4d4aa33ee9de7e090c79/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:36.255115 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c1d548-52d2-4040-94a4-46ab274ff635 None admin] RBAC: Authorizing `identity:create_grant(project_id=b08ea5fe820543b6936ffde78e21ea4f, user_id=de97b5d1390d4d4aa33ee9de7e090c79, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:36.256888 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c1d548-52d2-4040-94a4-46ab274ff635 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:36.274227 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 129/265] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 05:05:35 2022] PUT /identity/v3/projects/b08ea5fe820543b6936ffde78e21ea4f/users/de97b5d1390d4d4aa33ee9de7e090c79/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:05:36.275253 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c1d548-52d2-4040-94a4-46ab274ff635 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:36.279387 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:36.301703 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b64617-c36c-4a8a-89b8-c2a148a9782b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:36.383049 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dafebc-fea9-40b5-ab38-504d74735625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:36.384309 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dafebc-fea9-40b5-ab38-504d74735625 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:36.384582 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dafebc-fea9-40b5-ab38-504d74735625 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:36.385130 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dafebc-fea9-40b5-ab38-504d74735625 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:36.398371 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c1d548-52d2-4040-94a4-46ab274ff635 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:36.435929 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf476edb-77f5-4d60-9a2b-b59dede9648a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:36.436684 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf476edb-77f5-4d60-9a2b-b59dede9648a None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:36.436962 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf476edb-77f5-4d60-9a2b-b59dede9648a None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:36.437232 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf476edb-77f5-4d60-9a2b-b59dede9648a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:36.437971 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf476edb-77f5-4d60-9a2b-b59dede9648a None admin] RBAC: Adding query filter params (role.id=c3d5589dd2864be9a0f26f5802d874cd, scope.project.id=b08ea5fe820543b6936ffde78e21ea4f, user.id=eea7e5f8e74547a7bd6329f40a48dbc4) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:36.438308 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf476edb-77f5-4d60-9a2b-b59dede9648a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:36.439789 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf476edb-77f5-4d60-9a2b-b59dede9648a None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:36.446409 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 130/266] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:05:36 2022] GET /identity/v3/role_assignments?user.id=eea7e5f8e74547a7bd6329f40a48dbc4&scope.project.id=b08ea5fe820543b6936ffde78e21ea4f&role.id=c3d5589dd2864be9a0f26f5802d874cd => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:36.447690 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fccb69-6b8f-4c72-97c1-1429610512a5 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:36.447972 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fccb69-6b8f-4c72-97c1-1429610512a5 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:36.448243 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fccb69-6b8f-4c72-97c1-1429610512a5 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:36.448638 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:36.449453 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 131/267] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:36.455246 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4da9836-1968-4a2b-9b23-0cc6f4fe5fa1 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:36.455683 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4da9836-1968-4a2b-9b23-0cc6f4fe5fa1 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:36.456043 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4da9836-1968-4a2b-9b23-0cc6f4fe5fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:36.477234 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-c4da9836-1968-4a2b-9b23-0cc6f4fe5fa1 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:36.478425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dafebc-fea9-40b5-ab38-504d74735625 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:36.478757 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c4da9836-1968-4a2b-9b23-0cc6f4fe5fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:36.480242 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dafebc-fea9-40b5-ab38-504d74735625 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:36.567694 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:36.580987 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 132/268] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:36.617680 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 137/269] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:05:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 05:05:36.623422 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4da9836-1968-4a2b-9b23-0cc6f4fe5fa1 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:36.667855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aebbef-d8ca-4793-a09b-6d8e4ba48e48 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:36.668157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aebbef-d8ca-4793-a09b-6d8e4ba48e48 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:36.668459 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aebbef-d8ca-4793-a09b-6d8e4ba48e48 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:36.669940 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 138/270] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 05:05:36.676099 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03237fa0-da6e-4707-a5b5-c71928f704e0 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:36.676708 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03237fa0-da6e-4707-a5b5-c71928f704e0 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:36.677177 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03237fa0-da6e-4707-a5b5-c71928f704e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:36.701355 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-03237fa0-da6e-4707-a5b5-c71928f704e0 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:36.702984 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-03237fa0-da6e-4707-a5b5-c71928f704e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:36.770118 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4da9836-1968-4a2b-9b23-0cc6f4fe5fa1 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:36.817552 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d35561-e302-417c-ad84-56bfdf1bcc0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:36.818274 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d35561-e302-417c-ad84-56bfdf1bcc0e None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:36.818492 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d35561-e302-417c-ad84-56bfdf1bcc0e None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:36.818764 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d35561-e302-417c-ad84-56bfdf1bcc0e None admin] PATH_INFO: `/v3/projects/b08ea5fe820543b6936ffde78e21ea4f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:36.820574 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d35561-e302-417c-ad84-56bfdf1bcc0e None admin] RBAC: Authorizing `identity:get_project(project_id=b08ea5fe820543b6936ffde78e21ea4f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:36.822035 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d35561-e302-417c-ad84-56bfdf1bcc0e None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:36.823860 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 133/271] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:05:36 2022] GET /identity/v3/projects/b08ea5fe820543b6936ffde78e21ea4f => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:36.831407 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d35561-e302-417c-ad84-56bfdf1bcc0e None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:36.848613 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:36.863866 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 139/272] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 05:05:36.896660 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03237fa0-da6e-4707-a5b5-c71928f704e0 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:36.938402 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:36.980317 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d35561-e302-417c-ad84-56bfdf1bcc0e None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:37.024095 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca7039-a7be-4777-8242-41aa72148188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:37.025415 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca7039-a7be-4777-8242-41aa72148188 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:37.025889 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca7039-a7be-4777-8242-41aa72148188 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:37.026389 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca7039-a7be-4777-8242-41aa72148188 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:37.027315 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dca7039-a7be-4777-8242-41aa72148188 None admin] RBAC: Adding query filter params (role.id=a74bf01dd66f49f5988a197da269c822, scope.project.id=b08ea5fe820543b6936ffde78e21ea4f, user.id=eea7e5f8e74547a7bd6329f40a48dbc4) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:37.027820 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dca7039-a7be-4777-8242-41aa72148188 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:37.029454 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dca7039-a7be-4777-8242-41aa72148188 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:37.037057 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 134/273] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:05:36 2022] GET /identity/v3/role_assignments?user.id=eea7e5f8e74547a7bd6329f40a48dbc4&scope.project.id=b08ea5fe820543b6936ffde78e21ea4f&role.id=a74bf01dd66f49f5988a197da269c822 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:37.065707 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03237fa0-da6e-4707-a5b5-c71928f704e0 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:37.108768 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb02ac0e-a63e-4810-90da-510ffc9f9bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:37.109326 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02ac0e-a63e-4810-90da-510ffc9f9bca None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:37.109599 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02ac0e-a63e-4810-90da-510ffc9f9bca None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:37.109865 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02ac0e-a63e-4810-90da-510ffc9f9bca None admin] PATH_INFO: `/v3/roles/member` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:37.124866 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb02ac0e-a63e-4810-90da-510ffc9f9bca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:37.129754 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-bb02ac0e-a63e-4810-90da-510ffc9f9bca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 05:05:37.129754 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:37.129754 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:37.129754 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:37.129754 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:37.129754 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:37.129754 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:37.129754 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:37.129754 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:37.129754 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:37.129754 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:37.129754 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:37.129754 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:37.129754 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:37.129754 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:37.129754 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:37.130656 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:37.130656 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:37.130656 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:37.130656 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:37.130656 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:37.130656 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:37.130656 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:37.130656 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:37.130656 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:37.130656 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:37.130656 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:37.130656 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:37.130656 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:37.130656 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:37.130656 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:37.130656 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:37.131444 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:37.131444 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:37.131444 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:37.131444 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:37.131444 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:37.131444 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:37.131444 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:37.131444 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:37.131444 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:37.131444 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:37.131444 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 05:05:37.131444 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:05:37.131444 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 140/274] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:05:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:05:37.137019 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca7039-a7be-4777-8242-41aa72148188 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:37.236826 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb02ac0e-a63e-4810-90da-510ffc9f9bca None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:37.274676 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca7039-a7be-4777-8242-41aa72148188 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:37.310480 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:37.311999 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0061a5a-bebc-4ef3-a11f-2544dca2a20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:37.312652 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0061a5a-bebc-4ef3-a11f-2544dca2a20e None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:37.313177 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0061a5a-bebc-4ef3-a11f-2544dca2a20e None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:37.313455 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0061a5a-bebc-4ef3-a11f-2544dca2a20e None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:37.314170 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0061a5a-bebc-4ef3-a11f-2544dca2a20e None admin] RBAC: Adding query filter params (name=member) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:37.314475 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0061a5a-bebc-4ef3-a11f-2544dca2a20e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:37.316043 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0061a5a-bebc-4ef3-a11f-2544dca2a20e None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:37.332159 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 135/275] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 05:05:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:37.339865 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0061a5a-bebc-4ef3-a11f-2544dca2a20e None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:37.353822 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:37.403285 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb02ac0e-a63e-4810-90da-510ffc9f9bca None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:37.440348 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75551d18-64e2-4cd4-91f7-80526d5733fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:37.441311 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75551d18-64e2-4cd4-91f7-80526d5733fa None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:37.441770 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75551d18-64e2-4cd4-91f7-80526d5733fa None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:37.442235 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75551d18-64e2-4cd4-91f7-80526d5733fa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:37.444449 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75551d18-64e2-4cd4-91f7-80526d5733fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:37.446314 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75551d18-64e2-4cd4-91f7-80526d5733fa None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:37.448446 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 141/276] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:05:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:37.454087 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75551d18-64e2-4cd4-91f7-80526d5733fa None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:37.458977 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0061a5a-bebc-4ef3-a11f-2544dca2a20e None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:37.499039 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21edb8af-d537-43d0-a924-a9821301097d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:37.499677 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21edb8af-d537-43d0-a924-a9821301097d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:37.499958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21edb8af-d537-43d0-a924-a9821301097d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:37.500227 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21edb8af-d537-43d0-a924-a9821301097d None admin] PATH_INFO: `/v3/users/be74e1dc2a8b44f186608c46e93774b4` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:37.515887 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21edb8af-d537-43d0-a924-a9821301097d None admin] RBAC: Authorizing `identity:get_user(user_id=be74e1dc2a8b44f186608c46e93774b4)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:37.515887 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21edb8af-d537-43d0-a924-a9821301097d None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:37.517654 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 136/277] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:05:37 2022] GET /identity/v3/users/be74e1dc2a8b44f186608c46e93774b4 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:37.529119 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21edb8af-d537-43d0-a924-a9821301097d None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:37.586560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75551d18-64e2-4cd4-91f7-80526d5733fa None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:37.611768 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:37.624844 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d2b4ac-6558-43a6-84fa-14c2f9e85852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:37.625803 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2b4ac-6558-43a6-84fa-14c2f9e85852 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:37.627073 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2b4ac-6558-43a6-84fa-14c2f9e85852 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:37.627599 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2b4ac-6558-43a6-84fa-14c2f9e85852 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:37.628502 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d2b4ac-6558-43a6-84fa-14c2f9e85852 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:37.630304 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d2b4ac-6558-43a6-84fa-14c2f9e85852 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:37.630410 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:37.656071 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21edb8af-d537-43d0-a924-a9821301097d None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:37.666542 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 142/278] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Aug 31 05:05:37 2022] POST /identity/v3/users => generated 293 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:05:37.695894 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28ee0c2-854c-49d3-a3c2-e1da9021ae8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:37.696516 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ee0c2-854c-49d3-a3c2-e1da9021ae8d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:37.696834 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ee0c2-854c-49d3-a3c2-e1da9021ae8d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:37.697102 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ee0c2-854c-49d3-a3c2-e1da9021ae8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:37.697850 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28ee0c2-854c-49d3-a3c2-e1da9021ae8d None admin] RBAC: Adding query filter params (role.id=e3235eda7c654e2abf96ed6a494f5152, user.id=be74e1dc2a8b44f186608c46e93774b4, scope.system=all) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:37.698182 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28ee0c2-854c-49d3-a3c2-e1da9021ae8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:37.699689 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28ee0c2-854c-49d3-a3c2-e1da9021ae8d None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:37.706206 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 137/279] 10.222.0.36 () {56 vars in 1308 bytes} [Wed Aug 31 05:05:37 2022] GET /identity/v3/role_assignments?user.id=be74e1dc2a8b44f186608c46e93774b4&scope.system=all&role.id=e3235eda7c654e2abf96ed6a494f5152 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:37.726432 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7cee6-9861-47c5-ba96-2f704f83193e None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:37.726785 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7cee6-9861-47c5-ba96-2f704f83193e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:37.726982 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7cee6-9861-47c5-ba96-2f704f83193e None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:37.728715 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 143/280] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:37.734970 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d9e763-c944-49d8-ae8a-090762b29d62 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:37.735346 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d9e763-c944-49d8-ae8a-090762b29d62 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:37.735516 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d9e763-c944-49d8-ae8a-090762b29d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:37.755995 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-84d9e763-c944-49d8-ae8a-090762b29d62 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:37.757528 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-84d9e763-c944-49d8-ae8a-090762b29d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:37.837093 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 138/281] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:38.105556 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad7cee6-9861-47c5-ba96-2f704f83193e None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:38.216938 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad7cee6-9861-47c5-ba96-2f704f83193e None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:38.260111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d055355d-a87c-46f3-af1f-41b69132809f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:38.261134 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055355d-a87c-46f3-af1f-41b69132809f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:38.261606 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055355d-a87c-46f3-af1f-41b69132809f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:38.262123 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055355d-a87c-46f3-af1f-41b69132809f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:38.277492 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d055355d-a87c-46f3-af1f-41b69132809f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:38.283819 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-d055355d-a87c-46f3-af1f-41b69132809f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 05:05:38.283819 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:38.283819 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:38.283819 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:38.283819 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:38.283819 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:38.283819 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:38.283819 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:38.283819 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:38.283819 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:38.283819 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:38.283819 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:38.283819 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:38.283819 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:38.283819 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:38.283819 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:38.284751 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:38.284751 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:38.284751 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:38.284751 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:38.284751 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:38.284751 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:38.284751 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:38.284751 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:38.284751 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:38.284751 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:38.284751 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:38.284751 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:38.284751 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:38.284751 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:38.284751 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:38.284751 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:38.285530 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:38.285530 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:38.285530 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:38.285530 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:38.285530 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:38.285530 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:38.285530 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:38.285530 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:38.285530 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:38.285530 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:38.285530 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 05:05:38.285530 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:05:38.288231 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 144/282] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:05:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:05:38.294226 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d9e763-c944-49d8-ae8a-090762b29d62 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:38.348876 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:38.406415 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d9e763-c944-49d8-ae8a-090762b29d62 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:38.441138 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dc31ef-9bd0-4388-afe5-850412f4dbbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:38.441748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc31ef-9bd0-4388-afe5-850412f4dbbd None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:38.442036 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc31ef-9bd0-4388-afe5-850412f4dbbd None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:38.442317 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc31ef-9bd0-4388-afe5-850412f4dbbd None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:38.442982 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dc31ef-9bd0-4388-afe5-850412f4dbbd None admin] RBAC: Adding query filter params (name=reader) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:38.443278 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dc31ef-9bd0-4388-afe5-850412f4dbbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:38.444660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dc31ef-9bd0-4388-afe5-850412f4dbbd None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:38.453332 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 139/283] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 05:05:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:38.458909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d055355d-a87c-46f3-af1f-41b69132809f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:38.501939 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:38.573493 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d055355d-a87c-46f3-af1f-41b69132809f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:38.607720 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d036816-c6e2-4f97-b934-89ca65d95344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:38.608427 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d036816-c6e2-4f97-b934-89ca65d95344 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:38.608706 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d036816-c6e2-4f97-b934-89ca65d95344 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:38.608974 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d036816-c6e2-4f97-b934-89ca65d95344 None admin] PATH_INFO: `/v3/users/91a0b0b8f8174d7b80fe85ba36c32a11` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:38.610189 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d036816-c6e2-4f97-b934-89ca65d95344 None admin] RBAC: Authorizing `identity:get_user(user_id=91a0b0b8f8174d7b80fe85ba36c32a11)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:38.611775 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d036816-c6e2-4f97-b934-89ca65d95344 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:38.613531 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 145/284] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:05:38 2022] GET /identity/v3/users/91a0b0b8f8174d7b80fe85ba36c32a11 => generated 297 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:38.619254 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dc31ef-9bd0-4388-afe5-850412f4dbbd None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:38.722165 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dc31ef-9bd0-4388-afe5-850412f4dbbd None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:38.724392 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:38.756365 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060df513-2b34-438f-af2b-5e7ccb8c1ec2 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:38.756671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060df513-2b34-438f-af2b-5e7ccb8c1ec2 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:38.756905 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060df513-2b34-438f-af2b-5e7ccb8c1ec2 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:38.758500 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 146/285] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 05:05:38.760431 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e8f74-adec-43c2-8dfc-5c70c3c8cfb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:38.761129 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e8f74-adec-43c2-8dfc-5c70c3c8cfb4 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:38.761403 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e8f74-adec-43c2-8dfc-5c70c3c8cfb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:38.761667 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e8f74-adec-43c2-8dfc-5c70c3c8cfb4 None admin] PATH_INFO: `/v3/projects/b08ea5fe820543b6936ffde78e21ea4f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:38.763261 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4c6c9-de96-40ab-b2f1-a6e926d2585f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:38.763529 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4c6c9-de96-40ab-b2f1-a6e926d2585f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:38.763790 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4c6c9-de96-40ab-b2f1-a6e926d2585f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:38.770696 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5e8f74-adec-43c2-8dfc-5c70c3c8cfb4 None admin] RBAC: Authorizing `identity:get_project(project_id=b08ea5fe820543b6936ffde78e21ea4f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:38.773059 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5e8f74-adec-43c2-8dfc-5c70c3c8cfb4 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:38.784257 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 140/286] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:05:38 2022] GET /identity/v3/projects/b08ea5fe820543b6936ffde78e21ea4f => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:38.787458 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e8f74-adec-43c2-8dfc-5c70c3c8cfb4 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:38.800510 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-18c4c6c9-de96-40ab-b2f1-a6e926d2585f None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:38.801708 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-18c4c6c9-de96-40ab-b2f1-a6e926d2585f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:38.902924 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e8f74-adec-43c2-8dfc-5c70c3c8cfb4 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:38.909678 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 147/287] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 05:05:38.960117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c39948f-bf43-4bcc-a9fe-8b93fd5e2f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:38.960847 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c39948f-bf43-4bcc-a9fe-8b93fd5e2f5f None admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:38.961122 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c39948f-bf43-4bcc-a9fe-8b93fd5e2f5f None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:38.961440 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c39948f-bf43-4bcc-a9fe-8b93fd5e2f5f None admin] PATH_INFO: `/v3/projects/b08ea5fe820543b6936ffde78e21ea4f/users/91a0b0b8f8174d7b80fe85ba36c32a11/roles/c87ef585cc814d919ef69ba258ecbbbd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:38.966612 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c39948f-bf43-4bcc-a9fe-8b93fd5e2f5f None admin] RBAC: Authorizing `identity:create_grant(project_id=b08ea5fe820543b6936ffde78e21ea4f, user_id=91a0b0b8f8174d7b80fe85ba36c32a11, role_id=c87ef585cc814d919ef69ba258ecbbbd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:38.968106 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c39948f-bf43-4bcc-a9fe-8b93fd5e2f5f None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:38.995433 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 141/288] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 05:05:38 2022] PUT /identity/v3/projects/b08ea5fe820543b6936ffde78e21ea4f/users/91a0b0b8f8174d7b80fe85ba36c32a11/roles/c87ef585cc814d919ef69ba258ecbbbd => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:05:39.232630 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c4c6c9-de96-40ab-b2f1-a6e926d2585f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:39.276957 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:39.339292 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c4c6c9-de96-40ab-b2f1-a6e926d2585f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:39.386047 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eca0bb-80b0-4053-90d3-2a094e6def2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:39.386677 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eca0bb-80b0-4053-90d3-2a094e6def2e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:39.386955 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eca0bb-80b0-4053-90d3-2a094e6def2e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:39.387227 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eca0bb-80b0-4053-90d3-2a094e6def2e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:39.403247 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eca0bb-80b0-4053-90d3-2a094e6def2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:39.408201 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-24eca0bb-80b0-4053-90d3-2a094e6def2e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 05:05:39.408201 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:39.408201 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:39.408201 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:39.408201 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:39.408201 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:39.408201 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:39.408201 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:39.408201 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:39.408201 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:39.408201 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:39.408201 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:39.408201 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:39.408201 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:39.408201 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:39.408201 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:39.409104 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:39.409104 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:39.409104 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:39.409104 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:39.409104 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:39.409104 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:39.409104 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:39.409104 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:39.409104 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:39.409104 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:39.409104 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:39.409104 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:39.409104 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:39.409104 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:39.409104 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:39.409104 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:39.409981 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:39.409981 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:39.409981 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:39.409981 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:39.409981 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:39.409981 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:39.409981 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:39.409981 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:39.409981 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:39.409981 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:39.409981 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 05:05:39.409981 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:05:39.409981 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 148/289] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:05:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:05:39.419201 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c39948f-bf43-4bcc-a9fe-8b93fd5e2f5f None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:39.475285 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:39.568610 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c39948f-bf43-4bcc-a9fe-8b93fd5e2f5f None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:39.617562 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0122bb-3030-4f54-9bb1-021908d36a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:39.618217 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0122bb-3030-4f54-9bb1-021908d36a3a None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:39.618806 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0122bb-3030-4f54-9bb1-021908d36a3a None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:39.619076 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0122bb-3030-4f54-9bb1-021908d36a3a None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:39.619757 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0122bb-3030-4f54-9bb1-021908d36a3a None admin] RBAC: Adding query filter params (name=member) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:39.620059 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0122bb-3030-4f54-9bb1-021908d36a3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:39.621452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0122bb-3030-4f54-9bb1-021908d36a3a None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:39.630240 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 142/290] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 05:05:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:39.635882 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eca0bb-80b0-4053-90d3-2a094e6def2e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:39.736007 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:39.739795 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eca0bb-80b0-4053-90d3-2a094e6def2e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:39.775089 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad46775-7daa-46a7-8ada-f1c755bf905b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:39.775717 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad46775-7daa-46a7-8ada-f1c755bf905b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:39.775996 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad46775-7daa-46a7-8ada-f1c755bf905b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:39.776267 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad46775-7daa-46a7-8ada-f1c755bf905b None admin] PATH_INFO: `/v3/users/de97b5d1390d4d4aa33ee9de7e090c79` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:39.777413 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad46775-7daa-46a7-8ada-f1c755bf905b None admin] RBAC: Authorizing `identity:get_user(user_id=de97b5d1390d4d4aa33ee9de7e090c79)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:39.778893 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad46775-7daa-46a7-8ada-f1c755bf905b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:39.780811 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 149/291] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:05:39 2022] GET /identity/v3/users/de97b5d1390d4d4aa33ee9de7e090c79 => generated 297 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:39.786573 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0122bb-3030-4f54-9bb1-021908d36a3a None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:39.884020 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:39.891008 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0122bb-3030-4f54-9bb1-021908d36a3a None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:39.926541 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc140a-7a15-4f2c-b53d-cc42a31d0cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:39.927155 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc140a-7a15-4f2c-b53d-cc42a31d0cbd None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:39.927433 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc140a-7a15-4f2c-b53d-cc42a31d0cbd None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:39.927703 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc140a-7a15-4f2c-b53d-cc42a31d0cbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:39.996006 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fc140a-7a15-4f2c-b53d-cc42a31d0cbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:39.997497 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fc140a-7a15-4f2c-b53d-cc42a31d0cbd None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:40.006326 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ffc09-d5d1-4e1e-9e63-19d5c1597a50 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:40.006607 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ffc09-d5d1-4e1e-9e63-19d5c1597a50 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:40.006884 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ffc09-d5d1-4e1e-9e63-19d5c1597a50 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:40.008380 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 150/292] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:40.014534 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b733d6-d98f-4e8e-8f14-f81c10d3c135 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:40.014858 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b733d6-d98f-4e8e-8f14-f81c10d3c135 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:40.015126 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b733d6-d98f-4e8e-8f14-f81c10d3c135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:40.039080 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e1b733d6-d98f-4e8e-8f14-f81c10d3c135 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:40.040605 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e1b733d6-d98f-4e8e-8f14-f81c10d3c135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:40.121958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 143/293] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:05:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 05:05:40.126260 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:40.129031 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc140a-7a15-4f2c-b53d-cc42a31d0cbd None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:40.233417 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc140a-7a15-4f2c-b53d-cc42a31d0cbd None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:40.246626 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 151/294] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:40.255535 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55faeb4-45c5-42eb-b1cd-98043e23dc73 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:40.255981 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55faeb4-45c5-42eb-b1cd-98043e23dc73 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:40.256415 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55faeb4-45c5-42eb-b1cd-98043e23dc73 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:40.258108 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 152/295] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:40.271368 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a13426f-6849-4e5d-9a83-e7cfb803b0a8 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:40.271847 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a13426f-6849-4e5d-9a83-e7cfb803b0a8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:40.271958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a312c1d1-20cb-4d95-bd53-e79f1d7a66b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:40.272388 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a13426f-6849-4e5d-9a83-e7cfb803b0a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:40.272497 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a312c1d1-20cb-4d95-bd53-e79f1d7a66b2 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:40.272767 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a312c1d1-20cb-4d95-bd53-e79f1d7a66b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:40.273041 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a312c1d1-20cb-4d95-bd53-e79f1d7a66b2 None admin] PATH_INFO: `/v3/projects/b08ea5fe820543b6936ffde78e21ea4f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:40.274216 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a312c1d1-20cb-4d95-bd53-e79f1d7a66b2 None admin] RBAC: Authorizing `identity:get_project(project_id=b08ea5fe820543b6936ffde78e21ea4f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:40.275673 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a312c1d1-20cb-4d95-bd53-e79f1d7a66b2 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:40.282043 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 144/296] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:05:40 2022] GET /identity/v3/projects/b08ea5fe820543b6936ffde78e21ea4f => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:40.283386 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a312c1d1-20cb-4d95-bd53-e79f1d7a66b2 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:40.310986 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-3a13426f-6849-4e5d-9a83-e7cfb803b0a8 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:40.312510 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3a13426f-6849-4e5d-9a83-e7cfb803b0a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:40.350378 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:40.415649 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a312c1d1-20cb-4d95-bd53-e79f1d7a66b2 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:40.438490 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 153/297] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:40.440007 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687ac3a-593a-4f66-83e6-ada2120e0231 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:40.440470 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687ac3a-593a-4f66-83e6-ada2120e0231 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:40.440893 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687ac3a-593a-4f66-83e6-ada2120e0231 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:40.442585 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 154/298] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:40.460136 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8c4ecb-8466-4962-8425-2f901112892d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:40.461448 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c4ecb-8466-4962-8425-2f901112892d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:40.461898 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c4ecb-8466-4962-8425-2f901112892d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:40.463771 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e6448-2e6c-4fd1-9cd8-8277ba3b84f9 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:40.464288 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e6448-2e6c-4fd1-9cd8-8277ba3b84f9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:40.464785 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e6448-2e6c-4fd1-9cd8-8277ba3b84f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:40.470270 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c4ecb-8466-4962-8425-2f901112892d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:40.476658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8c4ecb-8466-4962-8425-2f901112892d None admin] RBAC: Adding query filter params (role.id=e3235eda7c654e2abf96ed6a494f5152, scope.project.id=b08ea5fe820543b6936ffde78e21ea4f, user.id=de97b5d1390d4d4aa33ee9de7e090c79) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:40.477281 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8c4ecb-8466-4962-8425-2f901112892d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:40.480719 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8c4ecb-8466-4962-8425-2f901112892d None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:40.494944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 145/299] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:05:40 2022] GET /identity/v3/role_assignments?user.id=de97b5d1390d4d4aa33ee9de7e090c79&scope.project.id=b08ea5fe820543b6936ffde78e21ea4f&role.id=e3235eda7c654e2abf96ed6a494f5152 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:05:40.496075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-246e6448-2e6c-4fd1-9cd8-8277ba3b84f9 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:40.497444 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-246e6448-2e6c-4fd1-9cd8-8277ba3b84f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:40.530478 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:40.570886 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 155/300] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 05:05:40.637843 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8c4ecb-8466-4962-8425-2f901112892d None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:40.762981 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8c4ecb-8466-4962-8425-2f901112892d None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:40.763617 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246e6448-2e6c-4fd1-9cd8-8277ba3b84f9 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:40.790471 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:40.803620 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a26f17-b6a3-4515-8231-a8ddbfd82fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:40.804261 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a26f17-b6a3-4515-8231-a8ddbfd82fcb None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:40.804540 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a26f17-b6a3-4515-8231-a8ddbfd82fcb None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:40.804807 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a26f17-b6a3-4515-8231-a8ddbfd82fcb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:40.819258 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a26f17-b6a3-4515-8231-a8ddbfd82fcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:40.825953 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-34a26f17-b6a3-4515-8231-a8ddbfd82fcb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 05:05:40.825953 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:40.825953 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:40.825953 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:40.825953 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:40.825953 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:40.825953 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:40.825953 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:40.825953 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:40.825953 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:40.825953 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:40.825953 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:40.825953 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:40.825953 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:40.825953 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:40.825953 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:40.826859 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:40.826859 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:40.826859 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:40.826859 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:40.826859 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:40.826859 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:40.826859 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:40.826859 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:40.826859 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:40.826859 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:40.826859 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:40.826859 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:40.826859 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:40.826859 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:40.826859 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:40.826859 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:40.827704 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:40.827704 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:40.827704 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:40.827704 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:40.827704 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:40.827704 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:40.827704 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:40.827704 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:40.827704 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:40.827704 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:40.827704 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 05:05:40.827704 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:05:40.830392 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 146/301] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:05:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:05:40.836152 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a26f17-b6a3-4515-8231-a8ddbfd82fcb None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:40.905860 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246e6448-2e6c-4fd1-9cd8-8277ba3b84f9 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:40.944575 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2eef0-9955-4c5b-a166-0c4bbcaaf670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:40.945219 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2eef0-9955-4c5b-a166-0c4bbcaaf670 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:40.945501 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2eef0-9955-4c5b-a166-0c4bbcaaf670 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:40.945767 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2eef0-9955-4c5b-a166-0c4bbcaaf670 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:40.956539 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f2eef0-9955-4c5b-a166-0c4bbcaaf670 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:40.961377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-72f2eef0-9955-4c5b-a166-0c4bbcaaf670 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 05:05:40.961377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:40.961377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:40.961377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:40.961377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:40.961377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:40.961377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:40.961377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:40.961377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:40.961377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:40.961377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:40.961377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:40.961377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:40.961377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:40.961377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:40.961377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:40.962311 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:40.962311 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:40.962311 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:40.962311 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:40.962311 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:40.962311 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:40.962311 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:40.962311 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:40.962311 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:40.962311 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:40.962311 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:40.962311 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:40.962311 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:40.962311 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:40.962311 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:40.962311 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:40.963118 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:40.963118 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:40.963118 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:40.963118 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:40.963118 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:40.963118 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:40.963118 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:40.963118 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:40.963118 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:40.963118 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:40.963118 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 05:05:40.963118 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:05:40.963118 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 156/302] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:05:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:05:40.963765 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2eef0-9955-4c5b-a166-0c4bbcaaf670 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:40.971632 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a26f17-b6a3-4515-8231-a8ddbfd82fcb None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:40.988192 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:41.005744 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107bdc77-df56-48a6-8020-b09d185838c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:41.006486 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107bdc77-df56-48a6-8020-b09d185838c6 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:41.006644 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107bdc77-df56-48a6-8020-b09d185838c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:41.006908 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107bdc77-df56-48a6-8020-b09d185838c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:41.007600 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107bdc77-df56-48a6-8020-b09d185838c6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:41.007932 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107bdc77-df56-48a6-8020-b09d185838c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:41.009252 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107bdc77-df56-48a6-8020-b09d185838c6 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:41.023128 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 147/303] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 05:05:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:41.023128 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107bdc77-df56-48a6-8020-b09d185838c6 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:41.072367 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:41.082809 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2eef0-9955-4c5b-a166-0c4bbcaaf670 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:41.117431 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f5642-614b-43cc-a5b4-079ef1605b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:41.118672 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f5642-614b-43cc-a5b4-079ef1605b97 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:41.118994 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f5642-614b-43cc-a5b4-079ef1605b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:41.119270 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f5642-614b-43cc-a5b4-079ef1605b97 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:41.120390 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2f5642-614b-43cc-a5b4-079ef1605b97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:41.121850 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2f5642-614b-43cc-a5b4-079ef1605b97 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:41.123659 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 157/304] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:05:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:41.126493 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107bdc77-df56-48a6-8020-b09d185838c6 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:41.128821 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f5642-614b-43cc-a5b4-079ef1605b97 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:41.165267 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8c2720-b3c6-4eef-921d-8e31a9316bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:41.166191 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c2720-b3c6-4eef-921d-8e31a9316bae None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:41.166646 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c2720-b3c6-4eef-921d-8e31a9316bae None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:41.167083 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c2720-b3c6-4eef-921d-8e31a9316bae None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:41.167952 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8c2720-b3c6-4eef-921d-8e31a9316bae None admin] RBAC: Adding query filter params (name=reader) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:41.168442 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8c2720-b3c6-4eef-921d-8e31a9316bae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:41.170534 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8c2720-b3c6-4eef-921d-8e31a9316bae None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:41.181711 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 148/305] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 05:05:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:41.183165 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8c2720-b3c6-4eef-921d-8e31a9316bae None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:41.214287 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:41.232566 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f5642-614b-43cc-a5b4-079ef1605b97 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:41.273779 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38250da3-9a69-4122-a618-f13175595036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:41.274431 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38250da3-9a69-4122-a618-f13175595036 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:41.275070 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38250da3-9a69-4122-a618-f13175595036 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:41.275070 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38250da3-9a69-4122-a618-f13175595036 None admin] PATH_INFO: `/v3/users/be74e1dc2a8b44f186608c46e93774b4` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:41.276013 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38250da3-9a69-4122-a618-f13175595036 None admin] RBAC: Authorizing `identity:get_user(user_id=be74e1dc2a8b44f186608c46e93774b4)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:41.277532 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38250da3-9a69-4122-a618-f13175595036 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:41.284145 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 158/306] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:05:41 2022] GET /identity/v3/users/be74e1dc2a8b44f186608c46e93774b4 => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:41.284145 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38250da3-9a69-4122-a618-f13175595036 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:41.307933 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8c2720-b3c6-4eef-921d-8e31a9316bae None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:41.320199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:41.345282 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d224dfc4-6397-4ad4-aeda-c55edb4d1e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:41.345894 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224dfc4-6397-4ad4-aeda-c55edb4d1e9d None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:41.346211 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224dfc4-6397-4ad4-aeda-c55edb4d1e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:41.346479 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224dfc4-6397-4ad4-aeda-c55edb4d1e9d None admin] PATH_INFO: `/v3/groups` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:41.347115 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d224dfc4-6397-4ad4-aeda-c55edb4d1e9d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:41.348560 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d224dfc4-6397-4ad4-aeda-c55edb4d1e9d None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:41.364432 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 149/307] 10.222.0.36 () {60 vars in 1134 bytes} [Wed Aug 31 05:05:41 2022] POST /identity/v3/groups => generated 229 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:05:41.365208 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d224dfc4-6397-4ad4-aeda-c55edb4d1e9d None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:41.383730 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38250da3-9a69-4122-a618-f13175595036 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:41.420373 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c7bb8-21bf-41c8-a25c-d9e0384cbf31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:41.421082 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c7bb8-21bf-41c8-a25c-d9e0384cbf31 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:41.421353 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c7bb8-21bf-41c8-a25c-d9e0384cbf31 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:41.421623 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c7bb8-21bf-41c8-a25c-d9e0384cbf31 None admin] PATH_INFO: `/v3/users/731652f9b2444871b9b42e87be881ea7` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:41.436236 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907c7bb8-21bf-41c8-a25c-d9e0384cbf31 None admin] RBAC: Authorizing `identity:get_user(user_id=731652f9b2444871b9b42e87be881ea7)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:41.437781 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907c7bb8-21bf-41c8-a25c-d9e0384cbf31 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:41.444621 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 159/308] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:05:41 2022] GET /identity/v3/users/731652f9b2444871b9b42e87be881ea7 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:41.444621 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a223d19-f3c2-4fd6-8f8e-c430766ce169 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:41.444621 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a223d19-f3c2-4fd6-8f8e-c430766ce169 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:41.444621 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a223d19-f3c2-4fd6-8f8e-c430766ce169 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:41.444621 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 160/309] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:41.446198 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a223d19-f3c2-4fd6-8f8e-c430766ce169 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:41.459444 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:41.503481 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d224dfc4-6397-4ad4-aeda-c55edb4d1e9d None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:41.543679 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b53ae-5b83-458d-85dd-51ead1295221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:41.544623 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b53ae-5b83-458d-85dd-51ead1295221 None admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:41.544904 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b53ae-5b83-458d-85dd-51ead1295221 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:41.545177 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b53ae-5b83-458d-85dd-51ead1295221 None admin] PATH_INFO: `/v3/system/users/be74e1dc2a8b44f186608c46e93774b4/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:41.546688 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05b53ae-5b83-458d-85dd-51ead1295221 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=be74e1dc2a8b44f186608c46e93774b4, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:41.548105 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05b53ae-5b83-458d-85dd-51ead1295221 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:41.558257 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 150/310] 10.222.0.36 () {58 vars in 1334 bytes} [Wed Aug 31 05:05:41 2022] PUT /identity/v3/system/users/be74e1dc2a8b44f186608c46e93774b4/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:05:41.559564 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae04743-865b-4d3a-bcf6-3ad78fd6800e None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:41.559893 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae04743-865b-4d3a-bcf6-3ad78fd6800e None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:41.560162 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae04743-865b-4d3a-bcf6-3ad78fd6800e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:41.571168 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a223d19-f3c2-4fd6-8f8e-c430766ce169 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:41.585899 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-7ae04743-865b-4d3a-bcf6-3ad78fd6800e None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:41.587581 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-7ae04743-865b-4d3a-bcf6-3ad78fd6800e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:41.612192 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd230be9-40a4-4a90-af81-da78809e26a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:41.613150 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd230be9-40a4-4a90-af81-da78809e26a6 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:41.613644 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd230be9-40a4-4a90-af81-da78809e26a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:41.614145 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd230be9-40a4-4a90-af81-da78809e26a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:41.615112 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd230be9-40a4-4a90-af81-da78809e26a6 None admin] RBAC: Adding query filter params (role.id=c87ef585cc814d919ef69ba258ecbbbd, user.id=731652f9b2444871b9b42e87be881ea7, scope.system=all) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:41.615635 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd230be9-40a4-4a90-af81-da78809e26a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:41.617310 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd230be9-40a4-4a90-af81-da78809e26a6 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:41.623678 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 161/311] 10.222.0.36 () {56 vars in 1308 bytes} [Wed Aug 31 05:05:41 2022] GET /identity/v3/role_assignments?user.id=731652f9b2444871b9b42e87be881ea7&scope.system=all&role.id=c87ef585cc814d919ef69ba258ecbbbd => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:05:41.624585 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:41.694782 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 151/312] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:41.963058 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd230be9-40a4-4a90-af81-da78809e26a6 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:42.045535 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:42.073310 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd230be9-40a4-4a90-af81-da78809e26a6 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:42.108638 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5824ff-4b53-4b19-a357-796167e8d091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:42.109268 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5824ff-4b53-4b19-a357-796167e8d091 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:42.109545 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5824ff-4b53-4b19-a357-796167e8d091 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:42.109815 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5824ff-4b53-4b19-a357-796167e8d091 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:42.120559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5824ff-4b53-4b19-a357-796167e8d091 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:42.125398 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-ed5824ff-4b53-4b19-a357-796167e8d091 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 05:05:42.125398 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:42.125398 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:42.125398 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:42.125398 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:42.125398 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:42.125398 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:42.125398 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:42.125398 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:42.125398 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:42.125398 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:42.125398 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:42.125398 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:42.125398 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:42.125398 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:42.125398 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:42.126303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:42.126303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:42.126303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:42.126303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:42.126303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:42.126303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:42.126303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:42.126303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:42.126303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:42.126303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:42.126303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:42.126303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:42.126303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:42.126303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:42.126303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:42.126303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:42.127111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:42.127111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:42.127111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:42.127111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:42.127111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:42.127111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:42.127111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:42.127111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:42.127111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:42.127111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:42.127111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 05:05:42.127111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:05:42.127111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 162/313] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:05:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:05:42.132668 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae04743-865b-4d3a-bcf6-3ad78fd6800e None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:42.236569 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae04743-865b-4d3a-bcf6-3ad78fd6800e None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:42.273069 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ce704-342e-4780-8afe-57422a625726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:42.273676 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ce704-342e-4780-8afe-57422a625726 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:42.273947 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ce704-342e-4780-8afe-57422a625726 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:42.274234 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ce704-342e-4780-8afe-57422a625726 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:42.274905 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7ce704-342e-4780-8afe-57422a625726 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:42.275200 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7ce704-342e-4780-8afe-57422a625726 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:42.276586 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7ce704-342e-4780-8afe-57422a625726 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:42.285351 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 152/314] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 05:05:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:42.290916 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5824ff-4b53-4b19-a357-796167e8d091 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:42.393875 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5824ff-4b53-4b19-a357-796167e8d091 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:42.428827 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6fe364-682c-48e1-9774-a7afd8b3311c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:42.429459 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6fe364-682c-48e1-9774-a7afd8b3311c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:42.429734 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6fe364-682c-48e1-9774-a7afd8b3311c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:42.430021 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6fe364-682c-48e1-9774-a7afd8b3311c None admin] PATH_INFO: `/v3/users/91a0b0b8f8174d7b80fe85ba36c32a11` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:42.431142 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6fe364-682c-48e1-9774-a7afd8b3311c None admin] RBAC: Authorizing `identity:get_user(user_id=91a0b0b8f8174d7b80fe85ba36c32a11)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:42.432593 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6fe364-682c-48e1-9774-a7afd8b3311c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:42.434277 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 163/315] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:05:42 2022] GET /identity/v3/users/91a0b0b8f8174d7b80fe85ba36c32a11 => generated 297 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:05:42.440155 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ce704-342e-4780-8afe-57422a625726 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:42.471368 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:42.549182 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ce704-342e-4780-8afe-57422a625726 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:42.584045 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5141d4ac-e9f2-4d6e-81d7-ed5aa3de4883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:42.584657 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5141d4ac-e9f2-4d6e-81d7-ed5aa3de4883 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:42.584935 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5141d4ac-e9f2-4d6e-81d7-ed5aa3de4883 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:42.585255 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5141d4ac-e9f2-4d6e-81d7-ed5aa3de4883 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:42.654028 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5141d4ac-e9f2-4d6e-81d7-ed5aa3de4883 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:42.655599 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5141d4ac-e9f2-4d6e-81d7-ed5aa3de4883 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:42.684450 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:42.763452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 153/316] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:05:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 05:05:42.769197 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6fe364-682c-48e1-9774-a7afd8b3311c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:42.772362 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:42.874805 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6fe364-682c-48e1-9774-a7afd8b3311c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:42.911532 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad2742-3346-455b-9085-16bb67181511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:42.912166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad2742-3346-455b-9085-16bb67181511 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:42.912443 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad2742-3346-455b-9085-16bb67181511 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:42.912713 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad2742-3346-455b-9085-16bb67181511 None admin] PATH_INFO: `/v3/projects/b08ea5fe820543b6936ffde78e21ea4f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:42.913825 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cad2742-3346-455b-9085-16bb67181511 None admin] RBAC: Authorizing `identity:get_project(project_id=b08ea5fe820543b6936ffde78e21ea4f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:42.915341 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cad2742-3346-455b-9085-16bb67181511 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:42.917295 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 164/317] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:05:42 2022] GET /identity/v3/projects/b08ea5fe820543b6936ffde78e21ea4f => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:42.923413 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5141d4ac-e9f2-4d6e-81d7-ed5aa3de4883 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:43.026809 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5141d4ac-e9f2-4d6e-81d7-ed5aa3de4883 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:43.053204 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:43.062344 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e1dd31-e0c4-4729-a11b-a699c052f70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:43.063101 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1dd31-e0c4-4729-a11b-a699c052f70c None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:43.063639 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1dd31-e0c4-4729-a11b-a699c052f70c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:43.064092 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1dd31-e0c4-4729-a11b-a699c052f70c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:43.064890 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e1dd31-e0c4-4729-a11b-a699c052f70c None admin] RBAC: Adding query filter params (role.id=c87ef585cc814d919ef69ba258ecbbbd, scope.project.id=b08ea5fe820543b6936ffde78e21ea4f, user.id=91a0b0b8f8174d7b80fe85ba36c32a11) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:43.065398 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e1dd31-e0c4-4729-a11b-a699c052f70c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:43.066874 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e1dd31-e0c4-4729-a11b-a699c052f70c None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:43.073392 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 154/318] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:05:42 2022] GET /identity/v3/role_assignments?user.id=91a0b0b8f8174d7b80fe85ba36c32a11&scope.project.id=b08ea5fe820543b6936ffde78e21ea4f&role.id=c87ef585cc814d919ef69ba258ecbbbd => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:43.289263 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbadba-c11e-488c-9240-399fa38ad06e None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:43.289629 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbadba-c11e-488c-9240-399fa38ad06e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:43.289816 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbadba-c11e-488c-9240-399fa38ad06e None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:43.291451 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 165/319] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:43.296864 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423efe93-7e47-45b8-96b7-a42c04fa4eba None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:43.297142 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423efe93-7e47-45b8-96b7-a42c04fa4eba None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:43.297415 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423efe93-7e47-45b8-96b7-a42c04fa4eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:43.315679 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-423efe93-7e47-45b8-96b7-a42c04fa4eba None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:43.316852 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-423efe93-7e47-45b8-96b7-a42c04fa4eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:43.385529 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 155/320] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:43.494261 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeefac7-fb6f-4ab0-be56-a4b8ae37499d None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:43.494629 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeefac7-fb6f-4ab0-be56-a4b8ae37499d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:43.494770 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeefac7-fb6f-4ab0-be56-a4b8ae37499d None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:43.496298 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 166/321] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:43.501410 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf8b21-c187-4ae8-8bcc-93646553f3e9 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:43.501694 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf8b21-c187-4ae8-8bcc-93646553f3e9 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:43.501964 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf8b21-c187-4ae8-8bcc-93646553f3e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:43.520595 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-d5bf8b21-c187-4ae8-8bcc-93646553f3e9 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:43.521758 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-d5bf8b21-c187-4ae8-8bcc-93646553f3e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:43.581884 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc484ead-c83c-4b42-a0a3-537ac45dc5e0 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:43.582086 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc484ead-c83c-4b42-a0a3-537ac45dc5e0 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:43.582374 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc484ead-c83c-4b42-a0a3-537ac45dc5e0 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:43.583865 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 167/322] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 05:05:43.588631 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cfef63-eded-40e3-af6c-cae307a76961 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:43.588915 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cfef63-eded-40e3-af6c-cae307a76961 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:43.589172 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cfef63-eded-40e3-af6c-cae307a76961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:43.595734 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 156/323] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:43.607223 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-53cfef63-eded-40e3-af6c-cae307a76961 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:43.608532 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-53cfef63-eded-40e3-af6c-cae307a76961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:43.647275 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf8b21-c187-4ae8-8bcc-93646553f3e9 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:43.677393 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:43.683479 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 168/324] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:43.701165 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:43.752418 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf8b21-c187-4ae8-8bcc-93646553f3e9 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:43.785979 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3653251-0754-441d-aeae-99fb42eea7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:43.786636 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3653251-0754-441d-aeae-99fb42eea7de None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:43.786909 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3653251-0754-441d-aeae-99fb42eea7de None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:43.787166 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3653251-0754-441d-aeae-99fb42eea7de None admin] PATH_INFO: `/v3/domains/default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:43.788222 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3653251-0754-441d-aeae-99fb42eea7de None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:43.789677 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3653251-0754-441d-aeae-99fb42eea7de None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:43.791438 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 157/325] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:05:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:05:43.796967 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cfef63-eded-40e3-af6c-cae307a76961 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:43.857936 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3653251-0754-441d-aeae-99fb42eea7de None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:43.899999 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cfef63-eded-40e3-af6c-cae307a76961 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:43.937894 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015c8049-51e4-40e7-8849-645b81adf426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:43.938538 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c8049-51e4-40e7-8849-645b81adf426 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:43.938806 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c8049-51e4-40e7-8849-645b81adf426 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:43.939071 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c8049-51e4-40e7-8849-645b81adf426 None admin] PATH_INFO: `/v3/groups` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:43.939705 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015c8049-51e4-40e7-8849-645b81adf426 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:43.941263 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015c8049-51e4-40e7-8849-645b81adf426 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:43.958631 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 169/326] 10.222.0.36 () {60 vars in 1134 bytes} [Wed Aug 31 05:05:43 2022] POST /identity/v3/groups => generated 226 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:05:43.959787 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015c8049-51e4-40e7-8849-645b81adf426 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:43.966329 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3653251-0754-441d-aeae-99fb42eea7de None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:43.988227 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:44.011178 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e4d05-f6aa-4f4c-8bdb-0fa8ba12cde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:44.011835 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e4d05-f6aa-4f4c-8bdb-0fa8ba12cde8 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:44.012105 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e4d05-f6aa-4f4c-8bdb-0fa8ba12cde8 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:44.012376 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e4d05-f6aa-4f4c-8bdb-0fa8ba12cde8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:44.025092 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397e4d05-f6aa-4f4c-8bdb-0fa8ba12cde8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:44.028019 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:44.029999 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-397e4d05-f6aa-4f4c-8bdb-0fa8ba12cde8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 05:05:44.029999 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:44.029999 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:44.029999 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:44.029999 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:44.029999 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:44.029999 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:44.029999 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:44.029999 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:44.029999 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:44.029999 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:44.029999 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:44.029999 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:44.029999 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:44.029999 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:44.029999 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:44.030981 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:44.030981 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:44.030981 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:44.030981 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:44.030981 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:44.030981 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:44.030981 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:44.030981 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:44.030981 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:44.030981 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:44.030981 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:44.030981 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:44.030981 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:44.030981 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:44.030981 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:44.030981 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:44.031783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:44.031783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:44.031783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:44.031783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:44.031783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:44.031783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:44.031783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:44.031783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:44.031783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:44.031783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:44.031783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 05:05:44.031783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:05:44.031783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 158/327] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:05:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:05:44.039102 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e4d05-f6aa-4f4c-8bdb-0fa8ba12cde8 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:44.081126 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015c8049-51e4-40e7-8849-645b81adf426 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:44.119080 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e52cb61-e762-40eb-a40d-acc4d80f5b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:44.119639 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52cb61-e762-40eb-a40d-acc4d80f5b33 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:44.119917 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52cb61-e762-40eb-a40d-acc4d80f5b33 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:44.120185 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52cb61-e762-40eb-a40d-acc4d80f5b33 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:44.130699 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e52cb61-e762-40eb-a40d-acc4d80f5b33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:44.135965 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-4e52cb61-e762-40eb-a40d-acc4d80f5b33 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 05:05:44.135965 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:44.135965 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:44.135965 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:44.135965 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:44.135965 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:44.135965 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:44.135965 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:44.135965 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:44.135965 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:44.135965 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:44.135965 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:44.135965 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:44.135965 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:44.135965 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:44.135965 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:44.136855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:44.136855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:44.136855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:44.136855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:44.136855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:44.136855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:44.136855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:44.136855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:44.136855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:44.136855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:44.136855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:44.136855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:44.136855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:44.136855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:44.136855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:44.136855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:44.137634 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:44.137634 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:44.137634 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:44.137634 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:44.137634 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:44.137634 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:44.137634 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:44.137634 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:44.137634 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:44.137634 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:44.137634 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 05:05:44.137634 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:05:44.137634 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 170/328] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:05:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:05:44.143866 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e52cb61-e762-40eb-a40d-acc4d80f5b33 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:44.151325 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e4d05-f6aa-4f4c-8bdb-0fa8ba12cde8 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:44.192857 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85ea96-a2a6-40c2-8781-3e104e2f93dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:44.193468 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85ea96-a2a6-40c2-8781-3e104e2f93dd None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:44.193742 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85ea96-a2a6-40c2-8781-3e104e2f93dd None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:44.194024 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85ea96-a2a6-40c2-8781-3e104e2f93dd None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:44.194707 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d85ea96-a2a6-40c2-8781-3e104e2f93dd None admin] RBAC: Adding query filter params (name=member) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:44.195009 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d85ea96-a2a6-40c2-8781-3e104e2f93dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:44.196433 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d85ea96-a2a6-40c2-8781-3e104e2f93dd None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:44.209656 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 159/329] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 05:05:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:44.215168 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85ea96-a2a6-40c2-8781-3e104e2f93dd None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:44.268163 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:44.270901 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e52cb61-e762-40eb-a40d-acc4d80f5b33 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:44.295168 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:44.321837 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed061265-0799-42f6-9505-b8e52cae46a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:44.322526 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed061265-0799-42f6-9505-b8e52cae46a6 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:44.322795 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed061265-0799-42f6-9505-b8e52cae46a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:44.323055 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed061265-0799-42f6-9505-b8e52cae46a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:44.323725 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed061265-0799-42f6-9505-b8e52cae46a6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:44.324018 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed061265-0799-42f6-9505-b8e52cae46a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:44.325387 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed061265-0799-42f6-9505-b8e52cae46a6 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:44.335566 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 171/330] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 05:05:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:05:44.341143 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed061265-0799-42f6-9505-b8e52cae46a6 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:44.350121 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85ea96-a2a6-40c2-8781-3e104e2f93dd None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:44.400828 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4246ce35-64b8-4760-813d-a5d805c060c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:44.402251 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246ce35-64b8-4760-813d-a5d805c060c0 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:44.402763 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246ce35-64b8-4760-813d-a5d805c060c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:44.403488 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246ce35-64b8-4760-813d-a5d805c060c0 None admin] PATH_INFO: `/v3/users/be74e1dc2a8b44f186608c46e93774b4` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:44.405144 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4246ce35-64b8-4760-813d-a5d805c060c0 None admin] RBAC: Authorizing `identity:get_user(user_id=be74e1dc2a8b44f186608c46e93774b4)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:44.407255 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4246ce35-64b8-4760-813d-a5d805c060c0 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:44.410145 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 160/331] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:05:44 2022] GET /identity/v3/users/be74e1dc2a8b44f186608c46e93774b4 => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:05:44.417282 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4246ce35-64b8-4760-813d-a5d805c060c0 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:44.467073 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed061265-0799-42f6-9505-b8e52cae46a6 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:44.510283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c9685e-1a01-431f-8c62-64ae2e605894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:44.511034 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c9685e-1a01-431f-8c62-64ae2e605894 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:44.511379 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c9685e-1a01-431f-8c62-64ae2e605894 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:44.511712 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c9685e-1a01-431f-8c62-64ae2e605894 None admin] PATH_INFO: `/v3/users/731652f9b2444871b9b42e87be881ea7` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:44.512976 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c9685e-1a01-431f-8c62-64ae2e605894 None admin] RBAC: Authorizing `identity:get_user(user_id=731652f9b2444871b9b42e87be881ea7)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:44.514508 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c9685e-1a01-431f-8c62-64ae2e605894 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:44.516392 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 172/332] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:05:44 2022] GET /identity/v3/users/731652f9b2444871b9b42e87be881ea7 => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:05:44.523610 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c9685e-1a01-431f-8c62-64ae2e605894 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:44.553657 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4246ce35-64b8-4760-813d-a5d805c060c0 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:44.564906 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:44.580191 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:44.599131 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b210b338-bec4-4e3f-bd70-25edbcc7c94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:44.599795 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b210b338-bec4-4e3f-bd70-25edbcc7c94b None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:44.600021 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b210b338-bec4-4e3f-bd70-25edbcc7c94b None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:44.600287 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b210b338-bec4-4e3f-bd70-25edbcc7c94b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:44.601024 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b210b338-bec4-4e3f-bd70-25edbcc7c94b None admin] RBAC: Adding query filter params (role.id=e3235eda7c654e2abf96ed6a494f5152, user.id=be74e1dc2a8b44f186608c46e93774b4, scope.system=all) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:44.601351 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b210b338-bec4-4e3f-bd70-25edbcc7c94b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:44.603379 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b210b338-bec4-4e3f-bd70-25edbcc7c94b None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:44.616973 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 161/333] 10.222.0.36 () {56 vars in 1308 bytes} [Wed Aug 31 05:05:44 2022] GET /identity/v3/role_assignments?user.id=be74e1dc2a8b44f186608c46e93774b4&scope.system=all&role.id=e3235eda7c654e2abf96ed6a494f5152 => generated 510 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:44.645748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c9685e-1a01-431f-8c62-64ae2e605894 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:44.683360 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae9257-0fbf-41e9-8752-72f54a7fe8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:44.683958 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae9257-0fbf-41e9-8752-72f54a7fe8d5 None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:44.684229 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae9257-0fbf-41e9-8752-72f54a7fe8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:44.684495 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae9257-0fbf-41e9-8752-72f54a7fe8d5 None admin] PATH_INFO: `/v3/system/users/731652f9b2444871b9b42e87be881ea7/roles/c87ef585cc814d919ef69ba258ecbbbd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:44.686032 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaae9257-0fbf-41e9-8752-72f54a7fe8d5 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=731652f9b2444871b9b42e87be881ea7, role_id=c87ef585cc814d919ef69ba258ecbbbd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:44.687530 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaae9257-0fbf-41e9-8752-72f54a7fe8d5 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:44.706192 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 173/334] 10.222.0.36 () {58 vars in 1334 bytes} [Wed Aug 31 05:05:44 2022] PUT /identity/v3/system/users/731652f9b2444871b9b42e87be881ea7/roles/c87ef585cc814d919ef69ba258ecbbbd => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:05:46.360271 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f88a6-7d30-4c86-9ec9-98bbcac6a061 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:46.360798 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f88a6-7d30-4c86-9ec9-98bbcac6a061 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:46.360798 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f88a6-7d30-4c86-9ec9-98bbcac6a061 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:46.362361 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 162/335] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:46.367814 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee765e-a994-4a45-a194-5405b8868993 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:46.368266 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee765e-a994-4a45-a194-5405b8868993 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:46.368654 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee765e-a994-4a45-a194-5405b8868993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:46.392687 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-2aee765e-a994-4a45-a194-5405b8868993 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:46.393886 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2aee765e-a994-4a45-a194-5405b8868993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:46.435801 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba26a51-0afb-46c5-970d-592dd17afae3 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:46.436179 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba26a51-0afb-46c5-970d-592dd17afae3 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:46.436495 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba26a51-0afb-46c5-970d-592dd17afae3 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:46.442104 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 163/336] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:46.447616 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34d669-b959-4f78-a619-ed2936d190c8 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:46.447895 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34d669-b959-4f78-a619-ed2936d190c8 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:46.448165 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34d669-b959-4f78-a619-ed2936d190c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:46.479602 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 174/337] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:46.483441 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-db34d669-b959-4f78-a619-ed2936d190c8 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:46.484658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-db34d669-b959-4f78-a619-ed2936d190c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:46.584615 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 164/338] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:46.840351 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee765e-a994-4a45-a194-5405b8868993 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:46.893796 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:46.959006 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee765e-a994-4a45-a194-5405b8868993 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:47.003743 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8da280-c33d-4adc-9e5d-8986c52d7780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:47.004373 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8da280-c33d-4adc-9e5d-8986c52d7780 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:47.004644 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8da280-c33d-4adc-9e5d-8986c52d7780 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:47.004908 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8da280-c33d-4adc-9e5d-8986c52d7780 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:47.008687 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1802cd-e53b-4313-9642-0f7f422329cc None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:47.009225 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1802cd-e53b-4313-9642-0f7f422329cc None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:47.009682 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1802cd-e53b-4313-9642-0f7f422329cc None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:47.011312 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 165/339] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:47.017101 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2286820e-25de-43cc-b82f-c3cecc69eb49 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:47.017625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2286820e-25de-43cc-b82f-c3cecc69eb49 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:47.018106 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2286820e-25de-43cc-b82f-c3cecc69eb49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:47.018488 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8da280-c33d-4adc-9e5d-8986c52d7780 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:47.023280 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-bf8da280-c33d-4adc-9e5d-8986c52d7780 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 05:05:47.023280 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:47.023280 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:47.023280 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:47.023280 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:47.023280 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:47.023280 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:47.023280 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:47.023280 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:47.023280 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:47.023280 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:47.023280 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:47.023280 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:47.023280 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:47.023280 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:47.023280 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:47.024153 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:47.024153 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:47.024153 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:47.024153 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:47.024153 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:47.024153 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:47.024153 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:47.024153 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:47.024153 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:47.024153 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:47.024153 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:47.024153 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:47.024153 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:47.024153 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:47.024153 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:47.024153 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:47.024951 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:47.024951 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:47.024951 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:47.024951 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:47.024951 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:47.024951 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:47.024951 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:47.024951 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:47.024951 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:47.024951 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:47.024951 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 05:05:47.024951 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:05:47.029014 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 175/340] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:05:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:05:47.030276 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8da280-c33d-4adc-9e5d-8986c52d7780 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:47.042617 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-2286820e-25de-43cc-b82f-c3cecc69eb49 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:47.044216 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-2286820e-25de-43cc-b82f-c3cecc69eb49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:47.118212 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 166/341] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 05:05:47.129417 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2286820e-25de-43cc-b82f-c3cecc69eb49 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:47.176907 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:47.244315 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8da280-c33d-4adc-9e5d-8986c52d7780 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:47.252861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:47.281888 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2286820e-25de-43cc-b82f-c3cecc69eb49 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:47.338721 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e13de31-6817-4714-ac71-d5617e90dd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:47.339586 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e13de31-6817-4714-ac71-d5617e90dd16 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:47.339998 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e13de31-6817-4714-ac71-d5617e90dd16 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:47.340404 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e13de31-6817-4714-ac71-d5617e90dd16 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:47.340643 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0aede-dfcf-4a40-be96-71bae190201f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:47.341347 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0aede-dfcf-4a40-be96-71bae190201f None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:47.341526 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e13de31-6817-4714-ac71-d5617e90dd16 None admin] RBAC: Adding query filter params (name=member) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:47.341744 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0aede-dfcf-4a40-be96-71bae190201f None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:47.342104 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0aede-dfcf-4a40-be96-71bae190201f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:47.342221 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e13de31-6817-4714-ac71-d5617e90dd16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:47.343769 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e13de31-6817-4714-ac71-d5617e90dd16 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:47.357209 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 176/342] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 05:05:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:05:47.359370 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb0aede-dfcf-4a40-be96-71bae190201f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:47.363348 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e13de31-6817-4714-ac71-d5617e90dd16 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:47.364548 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-3bb0aede-dfcf-4a40-be96-71bae190201f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 05:05:47.364548 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:47.364548 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:47.364548 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:47.364548 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:47.364548 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:47.364548 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:47.364548 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:47.364548 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:47.364548 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:47.364548 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:47.364548 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:47.364548 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:47.364548 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:47.364548 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:47.364548 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:47.365912 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:47.365912 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:47.365912 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:47.365912 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:47.365912 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:47.365912 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:47.365912 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:47.365912 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:47.365912 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:47.365912 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:47.365912 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:47.365912 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:47.365912 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:47.365912 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:47.365912 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:47.365912 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:47.366922 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:47.366922 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:47.366922 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:47.366922 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:47.366922 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:47.366922 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:47.366922 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:47.366922 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:47.366922 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:47.366922 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:47.366922 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 05:05:47.366922 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:05:47.367846 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 167/343] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 05:05:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:05:47.373424 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0aede-dfcf-4a40-be96-71bae190201f None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:47.489670 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:47.500882 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0aede-dfcf-4a40-be96-71bae190201f None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:47.539095 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f8a418-48c8-429b-b725-1e26f30f98a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:47.540133 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f8a418-48c8-429b-b725-1e26f30f98a6 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:47.540576 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f8a418-48c8-429b-b725-1e26f30f98a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:47.541010 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f8a418-48c8-429b-b725-1e26f30f98a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:47.541843 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f8a418-48c8-429b-b725-1e26f30f98a6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:47.542373 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f8a418-48c8-429b-b725-1e26f30f98a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:47.543925 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f8a418-48c8-429b-b725-1e26f30f98a6 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:47.553647 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 168/344] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:05:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:47.559115 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f8a418-48c8-429b-b725-1e26f30f98a6 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:47.571954 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e13de31-6817-4714-ac71-d5617e90dd16 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:47.626451 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8d738-e8fb-47ab-b2f3-c1f16fdf9dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:47.627388 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8d738-e8fb-47ab-b2f3-c1f16fdf9dfe None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:47.627828 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8d738-e8fb-47ab-b2f3-c1f16fdf9dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:47.628257 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8d738-e8fb-47ab-b2f3-c1f16fdf9dfe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:47.669777 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:47.684116 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f8a418-48c8-429b-b725-1e26f30f98a6 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:47.709977 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e8d738-e8fb-47ab-b2f3-c1f16fdf9dfe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:47.712665 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e8d738-e8fb-47ab-b2f3-c1f16fdf9dfe None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:47.728548 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa0ecca-9809-43cb-8c65-d2feb482d9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:47.729158 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa0ecca-9809-43cb-8c65-d2feb482d9a8 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:47.729432 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa0ecca-9809-43cb-8c65-d2feb482d9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:47.729701 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa0ecca-9809-43cb-8c65-d2feb482d9a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:47.754410 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:47.816572 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa0ecca-9809-43cb-8c65-d2feb482d9a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:47.818387 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa0ecca-9809-43cb-8c65-d2feb482d9a8 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:47.837798 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 177/345] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:05:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 05:05:47.839831 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c4c7b-3bf3-44bc-a7d3-8f2db31f48f5 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:47.840283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c4c7b-3bf3-44bc-a7d3-8f2db31f48f5 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:47.840715 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c4c7b-3bf3-44bc-a7d3-8f2db31f48f5 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:47.846492 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 178/346] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:47.847622 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c4c7b-3bf3-44bc-a7d3-8f2db31f48f5 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:47.925516 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:47.948918 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 169/347] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:05:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 05:05:47.949858 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa0ecca-9809-43cb-8c65-d2feb482d9a8 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:47.972727 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c4c7b-3bf3-44bc-a7d3-8f2db31f48f5 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:48.014556 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:48.021404 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a13a037-3e61-4c11-b6bb-4ef488c13784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:48.022112 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a13a037-3e61-4c11-b6bb-4ef488c13784 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:48.022390 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a13a037-3e61-4c11-b6bb-4ef488c13784 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:48.022822 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a13a037-3e61-4c11-b6bb-4ef488c13784 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:48.035381 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a13a037-3e61-4c11-b6bb-4ef488c13784 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:48.040885 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-7a13a037-3e61-4c11-b6bb-4ef488c13784 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 05:05:48.040885 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:48.040885 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:48.040885 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:48.040885 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:48.040885 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:48.040885 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:48.040885 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:48.040885 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:48.040885 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:48.040885 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:48.040885 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:48.040885 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:48.040885 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:48.040885 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:48.040885 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:48.041805 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:48.041805 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:48.041805 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:48.041805 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:48.041805 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:48.041805 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:48.041805 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:48.041805 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:48.041805 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:48.041805 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:48.041805 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:48.041805 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:48.041805 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:48.041805 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:48.041805 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:48.041805 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:48.042664 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:48.042664 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:48.042664 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:48.042664 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:48.042664 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:48.042664 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:48.042664 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:48.042664 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:48.042664 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:48.042664 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:48.042664 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 05:05:48.042664 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:05:48.044483 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 179/348] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:05:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:05:48.045900 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6087ee9-2f00-41e6-b559-8d22c95d8cb6 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:48.046334 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6087ee9-2f00-41e6-b559-8d22c95d8cb6 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:48.046749 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6087ee9-2f00-41e6-b559-8d22c95d8cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:48.065547 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b6087ee9-2f00-41e6-b559-8d22c95d8cb6 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:48.066841 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b6087ee9-2f00-41e6-b559-8d22c95d8cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:48.089664 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa0ecca-9809-43cb-8c65-d2feb482d9a8 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:48.136973 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb3e14f-2050-46dc-9366-e8de7329e5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:48.137721 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3e14f-2050-46dc-9366-e8de7329e5d3 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:48.137982 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3e14f-2050-46dc-9366-e8de7329e5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:48.138349 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3e14f-2050-46dc-9366-e8de7329e5d3 None admin] PATH_INFO: `/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:48.139512 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb3e14f-2050-46dc-9366-e8de7329e5d3 None admin] RBAC: Authorizing `identity:get_project(project_id=f9d86697e3eb4fd6b8f82a013cfcb5ba)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:48.141189 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb3e14f-2050-46dc-9366-e8de7329e5d3 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:48.142871 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 170/349] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:05:47 2022] GET /identity/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:48.147530 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb3e14f-2050-46dc-9366-e8de7329e5d3 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:48.149385 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 180/350] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:48.150705 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032eee80-8b2a-4326-b314-87119701fd4b None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:48.151832 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032eee80-8b2a-4326-b314-87119701fd4b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:48.151832 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032eee80-8b2a-4326-b314-87119701fd4b None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:48.152631 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 181/351] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:48.153193 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032eee80-8b2a-4326-b314-87119701fd4b None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:48.222408 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:48.274693 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032eee80-8b2a-4326-b314-87119701fd4b None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:48.285479 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb3e14f-2050-46dc-9366-e8de7329e5d3 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:48.306140 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:48.318997 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d435d4-f4d9-428e-911b-405906b82428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:48.319639 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d435d4-f4d9-428e-911b-405906b82428 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:48.320210 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d435d4-f4d9-428e-911b-405906b82428 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:48.320484 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d435d4-f4d9-428e-911b-405906b82428 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:48.321172 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d435d4-f4d9-428e-911b-405906b82428 None admin] RBAC: Adding query filter params (name=member) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:48.321486 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d435d4-f4d9-428e-911b-405906b82428 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:48.323642 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d435d4-f4d9-428e-911b-405906b82428 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:48.331477 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0391b8-a494-4926-aa20-87dc5591c878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:48.332458 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0391b8-a494-4926-aa20-87dc5591c878 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:48.332875 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0391b8-a494-4926-aa20-87dc5591c878 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:48.333328 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0391b8-a494-4926-aa20-87dc5591c878 None admin] PATH_INFO: `/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:48.334070 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 182/352] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 05:05:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:48.334860 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0391b8-a494-4926-aa20-87dc5591c878 None admin] RBAC: Authorizing `identity:get_project(project_id=f9d86697e3eb4fd6b8f82a013cfcb5ba)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:48.334972 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d435d4-f4d9-428e-911b-405906b82428 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:48.336639 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0391b8-a494-4926-aa20-87dc5591c878 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:48.343259 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 171/353] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:05:48 2022] GET /identity/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:48.346778 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de5734-c001-4fbd-9a9f-f658d471825e None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:48.347315 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de5734-c001-4fbd-9a9f-f658d471825e None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:48.347720 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de5734-c001-4fbd-9a9f-f658d471825e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:48.371430 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-16de5734-c001-4fbd-9a9f-f658d471825e None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:48.373682 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-16de5734-c001-4fbd-9a9f-f658d471825e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:48.469804 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 172/354] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:48.470859 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16de5734-c001-4fbd-9a9f-f658d471825e None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:48.486555 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d435d4-f4d9-428e-911b-405906b82428 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:48.508076 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:48.527789 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faa2bbb-91a0-498d-8065-c3c5dc3d4206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:48.528318 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa2bbb-91a0-498d-8065-c3c5dc3d4206 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:48.528591 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa2bbb-91a0-498d-8065-c3c5dc3d4206 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:48.528855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa2bbb-91a0-498d-8065-c3c5dc3d4206 None admin] PATH_INFO: `/v3/groups/89d78f08484744609eb0db0cdb5c9d27` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:48.535613 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faa2bbb-91a0-498d-8065-c3c5dc3d4206 None admin] RBAC: Authorizing `identity:get_group(group_id=89d78f08484744609eb0db0cdb5c9d27)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:48.537119 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faa2bbb-91a0-498d-8065-c3c5dc3d4206 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:48.539157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 183/355] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Aug 31 05:05:48 2022] GET /identity/v3/groups/89d78f08484744609eb0db0cdb5c9d27 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:48.539929 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faa2bbb-91a0-498d-8065-c3c5dc3d4206 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:48.603004 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16de5734-c001-4fbd-9a9f-f658d471825e None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:48.621769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:48.657030 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e4d9e-e4ab-493f-8c50-a0c1eceec141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:48.658094 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e4d9e-e4ab-493f-8c50-a0c1eceec141 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:48.658795 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e4d9e-e4ab-493f-8c50-a0c1eceec141 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:48.659457 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e4d9e-e4ab-493f-8c50-a0c1eceec141 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:48.661041 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faa2bbb-91a0-498d-8065-c3c5dc3d4206 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:48.701520 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d35f5ae-4cbe-434b-bff8-d0285442d02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:48.702261 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35f5ae-4cbe-434b-bff8-d0285442d02d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:48.702542 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35f5ae-4cbe-434b-bff8-d0285442d02d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:48.702815 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35f5ae-4cbe-434b-bff8-d0285442d02d None admin] PATH_INFO: `/v3/groups/89d78f08484744609eb0db0cdb5c9d27` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:48.704109 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d35f5ae-4cbe-434b-bff8-d0285442d02d None admin] RBAC: Authorizing `identity:get_group(group_id=89d78f08484744609eb0db0cdb5c9d27)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:48.706451 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d35f5ae-4cbe-434b-bff8-d0285442d02d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:48.708350 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 184/356] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Aug 31 05:05:48 2022] GET /identity/v3/groups/89d78f08484744609eb0db0cdb5c9d27 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:48.709122 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d35f5ae-4cbe-434b-bff8-d0285442d02d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:48.754985 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e4d9e-e4ab-493f-8c50-a0c1eceec141 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:48.756848 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e4d9e-e4ab-493f-8c50-a0c1eceec141 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:48.777947 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:48.843700 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d35f5ae-4cbe-434b-bff8-d0285442d02d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:48.880989 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 173/357] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:05:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 05:05:48.881917 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e4d9e-e4ab-493f-8c50-a0c1eceec141 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:48.885056 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06139860-1a29-4ba7-93ab-05bea14918e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:48.885760 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06139860-1a29-4ba7-93ab-05bea14918e2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:48.886071 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06139860-1a29-4ba7-93ab-05bea14918e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:48.886350 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06139860-1a29-4ba7-93ab-05bea14918e2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:48.897684 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06139860-1a29-4ba7-93ab-05bea14918e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:48.899822 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:48.904087 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-06139860-1a29-4ba7-93ab-05bea14918e2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 05:05:48.904087 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:48.904087 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:48.904087 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:48.904087 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:48.904087 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:48.904087 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:48.904087 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:48.904087 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:48.904087 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:48.904087 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:48.904087 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:48.904087 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:48.904087 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:48.904087 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:48.904087 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:48.905004 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:48.905004 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:48.905004 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:48.905004 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:48.905004 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:48.905004 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:48.905004 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:48.905004 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:48.905004 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:48.905004 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:48.905004 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:48.905004 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:48.905004 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:48.905004 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:48.905004 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:48.905004 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:48.909807 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:48.909807 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:48.909807 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:48.909807 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:48.909807 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:48.909807 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:48.909807 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:48.909807 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:48.909807 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:48.909807 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:48.909807 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 05:05:48.909807 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:05:48.909807 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 185/358] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:05:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:05:48.909807 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b50994a-07a3-4292-be55-0c2d7c848a98 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:48.910563 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b50994a-07a3-4292-be55-0c2d7c848a98 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:48.910563 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b50994a-07a3-4292-be55-0c2d7c848a98 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:48.910563 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 186/359] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:48.910563 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b50994a-07a3-4292-be55-0c2d7c848a98 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:49.006292 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e4d9e-e4ab-493f-8c50-a0c1eceec141 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:49.024464 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b50994a-07a3-4292-be55-0c2d7c848a98 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:49.035613 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:49.052690 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfedcdf2-42c8-4e35-a0fe-6d81e5e8c328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:49.053304 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfedcdf2-42c8-4e35-a0fe-6d81e5e8c328 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:49.053590 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfedcdf2-42c8-4e35-a0fe-6d81e5e8c328 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:49.053838 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfedcdf2-42c8-4e35-a0fe-6d81e5e8c328 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:49.054591 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfedcdf2-42c8-4e35-a0fe-6d81e5e8c328 None admin] RBAC: Adding query filter params (group.id=89d78f08484744609eb0db0cdb5c9d27, role.id=e3235eda7c654e2abf96ed6a494f5152, scope.project.id=f9d86697e3eb4fd6b8f82a013cfcb5ba) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:49.054890 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfedcdf2-42c8-4e35-a0fe-6d81e5e8c328 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:49.057427 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfedcdf2-42c8-4e35-a0fe-6d81e5e8c328 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:49.066785 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 174/360] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Aug 31 05:05:48 2022] GET /identity/v3/role_assignments?group.id=89d78f08484744609eb0db0cdb5c9d27&scope.project.id=f9d86697e3eb4fd6b8f82a013cfcb5ba&role.id=e3235eda7c654e2abf96ed6a494f5152 => generated 260 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:49.067028 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab485a-3efe-4e66-9edb-866ed38aac10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:49.067028 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab485a-3efe-4e66-9edb-866ed38aac10 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:49.067332 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab485a-3efe-4e66-9edb-866ed38aac10 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:49.067470 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfedcdf2-42c8-4e35-a0fe-6d81e5e8c328 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:49.067634 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab485a-3efe-4e66-9edb-866ed38aac10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:49.068068 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab485a-3efe-4e66-9edb-866ed38aac10 None admin] RBAC: Adding query filter params (group.id=89d78f08484744609eb0db0cdb5c9d27, role.id=a74bf01dd66f49f5988a197da269c822, scope.project.id=f9d86697e3eb4fd6b8f82a013cfcb5ba) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:49.068375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab485a-3efe-4e66-9edb-866ed38aac10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:49.069890 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab485a-3efe-4e66-9edb-866ed38aac10 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:49.086113 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 187/361] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Aug 31 05:05:48 2022] GET /identity/v3/role_assignments?group.id=89d78f08484744609eb0db0cdb5c9d27&scope.project.id=f9d86697e3eb4fd6b8f82a013cfcb5ba&role.id=a74bf01dd66f49f5988a197da269c822 => generated 260 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:49.087611 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab485a-3efe-4e66-9edb-866ed38aac10 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:49.187271 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:49.224159 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfedcdf2-42c8-4e35-a0fe-6d81e5e8c328 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:49.257056 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab485a-3efe-4e66-9edb-866ed38aac10 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:49.284021 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee73c585-c317-4f93-a925-d127266f17f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:49.284666 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73c585-c317-4f93-a925-d127266f17f9 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:49.284944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73c585-c317-4f93-a925-d127266f17f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:49.285212 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73c585-c317-4f93-a925-d127266f17f9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:49.301510 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee73c585-c317-4f93-a925-d127266f17f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:49.308549 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-ee73c585-c317-4f93-a925-d127266f17f9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 05:05:49.308549 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:49.308549 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:49.308549 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:49.308549 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:49.308549 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:49.308549 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:49.308549 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:49.308549 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:49.308549 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:49.308549 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:49.308549 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:49.308549 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:49.308549 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:49.308549 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:49.308549 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:49.309655 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:49.309655 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:49.309655 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:49.309655 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:49.309655 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:49.309655 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:49.309655 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:49.309655 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:49.309655 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:49.309655 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:49.309655 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:49.309655 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:49.309655 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:49.309655 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:49.309655 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:49.309655 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:49.310642 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:49.310642 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:49.310642 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:49.310642 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:49.310642 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:49.310642 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:49.310642 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:49.310642 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:49.310642 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:49.310642 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:49.310642 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 05:05:49.310642 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:05:49.311288 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaadffd-a4bc-4c34-a5cf-b8b114bb424c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:49.311288 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaadffd-a4bc-4c34-a5cf-b8b114bb424c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:49.311288 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaadffd-a4bc-4c34-a5cf-b8b114bb424c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:49.311656 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaadffd-a4bc-4c34-a5cf-b8b114bb424c None admin] PATH_INFO: `/v3/projects/b08ea5fe820543b6936ffde78e21ea4f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:49.313305 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 175/362] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 05:05:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:05:49.314370 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee73c585-c317-4f93-a925-d127266f17f9 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:49.318747 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaadffd-a4bc-4c34-a5cf-b8b114bb424c None admin] RBAC: Authorizing `identity:get_project(project_id=b08ea5fe820543b6936ffde78e21ea4f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:49.320322 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaadffd-a4bc-4c34-a5cf-b8b114bb424c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:49.322294 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 188/363] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:05:49 2022] GET /identity/v3/projects/b08ea5fe820543b6936ffde78e21ea4f => generated 302 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:49.323462 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b81d7-91fe-405b-8e47-60d2201ad959 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:49.323843 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b81d7-91fe-405b-8e47-60d2201ad959 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:49.324183 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b81d7-91fe-405b-8e47-60d2201ad959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:49.346939 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-357b81d7-91fe-405b-8e47-60d2201ad959 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:49.348261 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-357b81d7-91fe-405b-8e47-60d2201ad959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:49.390844 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:49.434062 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 189/364] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:49.434062 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357b81d7-91fe-405b-8e47-60d2201ad959 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:49.457370 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee73c585-c317-4f93-a925-d127266f17f9 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:49.495652 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c88083-dfed-458d-b0be-1abc0da4ad79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:49.496268 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c88083-dfed-458d-b0be-1abc0da4ad79 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:49.496548 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c88083-dfed-458d-b0be-1abc0da4ad79 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:49.496817 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c88083-dfed-458d-b0be-1abc0da4ad79 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:49.497491 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c88083-dfed-458d-b0be-1abc0da4ad79 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:49.497794 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c88083-dfed-458d-b0be-1abc0da4ad79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:49.499270 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c88083-dfed-458d-b0be-1abc0da4ad79 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:49.508304 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:49.510712 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 176/365] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 05:05:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:49.518348 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c88083-dfed-458d-b0be-1abc0da4ad79 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:49.631236 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357b81d7-91fe-405b-8e47-60d2201ad959 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:49.694480 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b779394-345e-4afb-bba0-9a311e3ce698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:49.695904 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b779394-345e-4afb-bba0-9a311e3ce698 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:49.696428 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b779394-345e-4afb-bba0-9a311e3ce698 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:49.696887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b779394-345e-4afb-bba0-9a311e3ce698 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:49.697944 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b779394-345e-4afb-bba0-9a311e3ce698 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:49.699254 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b779394-345e-4afb-bba0-9a311e3ce698 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:49.702747 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b779394-345e-4afb-bba0-9a311e3ce698 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:49.716336 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c88083-dfed-458d-b0be-1abc0da4ad79 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:49.718933 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 190/366] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:05:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:49.721151 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b779394-345e-4afb-bba0-9a311e3ce698 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:49.775101 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c06dc1-fb44-4fcd-8943-62b4dd7fc132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:49.777587 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c06dc1-fb44-4fcd-8943-62b4dd7fc132 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:49.777587 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c06dc1-fb44-4fcd-8943-62b4dd7fc132 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:49.777587 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c06dc1-fb44-4fcd-8943-62b4dd7fc132 None admin] PATH_INFO: `/v3/groups/89d78f08484744609eb0db0cdb5c9d27` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:49.777587 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c06dc1-fb44-4fcd-8943-62b4dd7fc132 None admin] RBAC: Authorizing `identity:get_group(group_id=89d78f08484744609eb0db0cdb5c9d27)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:49.780844 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c06dc1-fb44-4fcd-8943-62b4dd7fc132 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:49.783324 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 177/367] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Aug 31 05:05:49 2022] GET /identity/v3/groups/89d78f08484744609eb0db0cdb5c9d27 => generated 226 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:49.785870 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c06dc1-fb44-4fcd-8943-62b4dd7fc132 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:49.798366 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:49.870401 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b779394-345e-4afb-bba0-9a311e3ce698 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:49.871175 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:49.909526 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c06dc1-fb44-4fcd-8943-62b4dd7fc132 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:49.911183 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319cf790-a8ac-4446-9990-a9a19c1f3295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:49.911927 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319cf790-a8ac-4446-9990-a9a19c1f3295 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:49.912210 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319cf790-a8ac-4446-9990-a9a19c1f3295 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:49.912479 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319cf790-a8ac-4446-9990-a9a19c1f3295 None admin] PATH_INFO: `/v3/users/731652f9b2444871b9b42e87be881ea7` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:49.913618 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319cf790-a8ac-4446-9990-a9a19c1f3295 None admin] RBAC: Authorizing `identity:get_user(user_id=731652f9b2444871b9b42e87be881ea7)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:49.915124 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319cf790-a8ac-4446-9990-a9a19c1f3295 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:49.921999 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 191/368] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:05:49 2022] GET /identity/v3/users/731652f9b2444871b9b42e87be881ea7 => generated 293 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:49.922998 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319cf790-a8ac-4446-9990-a9a19c1f3295 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:49.945424 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a497dc6e-4576-4359-840a-8cf8a3a50bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:49.946058 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497dc6e-4576-4359-840a-8cf8a3a50bf3 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:49.946340 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497dc6e-4576-4359-840a-8cf8a3a50bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:49.946748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497dc6e-4576-4359-840a-8cf8a3a50bf3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:50.024975 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a497dc6e-4576-4359-840a-8cf8a3a50bf3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:50.026630 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a497dc6e-4576-4359-840a-8cf8a3a50bf3 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:50.029866 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319cf790-a8ac-4446-9990-a9a19c1f3295 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:50.031515 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:50.068259 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af79fb-d747-4ddb-879c-f2a372a3ca7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:50.068898 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af79fb-d747-4ddb-879c-f2a372a3ca7c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:50.069175 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af79fb-d747-4ddb-879c-f2a372a3ca7c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:50.069551 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af79fb-d747-4ddb-879c-f2a372a3ca7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:50.080687 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af79fb-d747-4ddb-879c-f2a372a3ca7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:50.087383 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-68af79fb-d747-4ddb-879c-f2a372a3ca7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:05:50.087383 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:50.087383 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:50.087383 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:50.087383 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:50.087383 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:50.087383 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:50.087383 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:50.087383 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:50.087383 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:50.087383 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:50.087383 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:50.087383 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:50.087383 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:50.087383 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:50.087383 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:50.088276 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:50.088276 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:50.088276 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:50.088276 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:50.088276 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:50.088276 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:50.088276 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:50.088276 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:50.088276 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:50.088276 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:50.088276 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:50.088276 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:50.088276 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:50.088276 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:50.088276 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:50.088276 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:50.089093 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:50.089093 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:50.089093 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:50.089093 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:50.089093 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:50.089093 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:50.089093 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:50.089093 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:50.089093 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:50.089093 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:50.089093 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:05:50.089093 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:05:50.093619 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 192/369] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 05:05:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:05:50.094681 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af79fb-d747-4ddb-879c-f2a372a3ca7c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:50.098107 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:50.137570 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 178/370] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:05:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 05:05:50.138494 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a497dc6e-4576-4359-840a-8cf8a3a50bf3 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:50.205258 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af79fb-d747-4ddb-879c-f2a372a3ca7c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:50.241261 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf0271-28e0-4fa4-a61d-31a4a6354dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:50.241877 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf0271-28e0-4fa4-a61d-31a4a6354dd0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:50.242199 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf0271-28e0-4fa4-a61d-31a4a6354dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:50.242299 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a497dc6e-4576-4359-840a-8cf8a3a50bf3 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:50.242460 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf0271-28e0-4fa4-a61d-31a4a6354dd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:50.243196 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daf0271-28e0-4fa4-a61d-31a4a6354dd0 None admin] RBAC: Adding query filter params (group.id=89d78f08484744609eb0db0cdb5c9d27, role.id=e3235eda7c654e2abf96ed6a494f5152, scope.project.id=b08ea5fe820543b6936ffde78e21ea4f) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:50.243815 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daf0271-28e0-4fa4-a61d-31a4a6354dd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:50.245337 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daf0271-28e0-4fa4-a61d-31a4a6354dd0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:50.251360 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 193/371] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Aug 31 05:05:50 2022] GET /identity/v3/role_assignments?group.id=89d78f08484744609eb0db0cdb5c9d27&scope.project.id=b08ea5fe820543b6936ffde78e21ea4f&role.id=e3235eda7c654e2abf96ed6a494f5152 => generated 260 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:50.252045 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf0271-28e0-4fa4-a61d-31a4a6354dd0 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:50.281989 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ab2c7d-367b-4963-9bdf-280691ffc402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:50.282640 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab2c7d-367b-4963-9bdf-280691ffc402 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:50.282913 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab2c7d-367b-4963-9bdf-280691ffc402 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:50.283180 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab2c7d-367b-4963-9bdf-280691ffc402 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:50.283616 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:50.283922 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ab2c7d-367b-4963-9bdf-280691ffc402 None admin] RBAC: Adding query filter params (role.id=c87ef585cc814d919ef69ba258ecbbbd, user.id=731652f9b2444871b9b42e87be881ea7, scope.system=all) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:50.284225 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ab2c7d-367b-4963-9bdf-280691ffc402 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:50.286095 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ab2c7d-367b-4963-9bdf-280691ffc402 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:50.301732 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 179/372] 10.222.0.36 () {56 vars in 1308 bytes} [Wed Aug 31 05:05:50 2022] GET /identity/v3/role_assignments?user.id=731652f9b2444871b9b42e87be881ea7&scope.system=all&role.id=c87ef585cc814d919ef69ba258ecbbbd => generated 510 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:50.306848 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ab2c7d-367b-4963-9bdf-280691ffc402 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:50.345588 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:50.367541 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf0271-28e0-4fa4-a61d-31a4a6354dd0 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:50.408141 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea7efe-e1c9-453b-a6f4-c5efcc3d67ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:50.408927 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea7efe-e1c9-453b-a6f4-c5efcc3d67ee None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:50.409345 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea7efe-e1c9-453b-a6f4-c5efcc3d67ee None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:50.409685 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea7efe-e1c9-453b-a6f4-c5efcc3d67ee None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:50.410598 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaea7efe-e1c9-453b-a6f4-c5efcc3d67ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:50.410910 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaea7efe-e1c9-453b-a6f4-c5efcc3d67ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:50.412337 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaea7efe-e1c9-453b-a6f4-c5efcc3d67ee None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:50.426460 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 194/373] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:05:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:50.429212 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea7efe-e1c9-453b-a6f4-c5efcc3d67ee None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:50.543206 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:50.553609 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea7efe-e1c9-453b-a6f4-c5efcc3d67ee None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:50.555479 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ab2c7d-367b-4963-9bdf-280691ffc402 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:50.593869 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6858d090-e4a4-4e4f-92c7-63597bb4c9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:50.594528 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858d090-e4a4-4e4f-92c7-63597bb4c9f6 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:50.594972 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858d090-e4a4-4e4f-92c7-63597bb4c9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:50.595251 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858d090-e4a4-4e4f-92c7-63597bb4c9f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:50.616568 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bf8335-3afc-4653-823e-dad6e7ef16a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:50.617206 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf8335-3afc-4653-823e-dad6e7ef16a0 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:50.617484 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf8335-3afc-4653-823e-dad6e7ef16a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:50.617755 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf8335-3afc-4653-823e-dad6e7ef16a0 None admin] PATH_INFO: `/v3/projects/b08ea5fe820543b6936ffde78e21ea4f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:50.619246 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bf8335-3afc-4653-823e-dad6e7ef16a0 None admin] RBAC: Authorizing `identity:get_project(project_id=b08ea5fe820543b6936ffde78e21ea4f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:50.620708 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bf8335-3afc-4653-823e-dad6e7ef16a0 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:50.623264 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 180/374] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:05:50 2022] GET /identity/v3/projects/b08ea5fe820543b6936ffde78e21ea4f => generated 302 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:50.629360 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bf8335-3afc-4653-823e-dad6e7ef16a0 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:50.683928 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6858d090-e4a4-4e4f-92c7-63597bb4c9f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:50.686783 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6858d090-e4a4-4e4f-92c7-63597bb4c9f6 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:50.759448 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:50.766256 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bf8335-3afc-4653-823e-dad6e7ef16a0 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:50.802473 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:50.822819 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb96d17-3c62-437d-9792-6c472723b619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:50.823475 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb96d17-3c62-437d-9792-6c472723b619 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:50.823755 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb96d17-3c62-437d-9792-6c472723b619 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:50.824028 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb96d17-3c62-437d-9792-6c472723b619 None admin] PATH_INFO: `/v3/groups/89d78f08484744609eb0db0cdb5c9d27` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:50.826154 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb96d17-3c62-437d-9792-6c472723b619 None admin] RBAC: Authorizing `identity:get_group(group_id=89d78f08484744609eb0db0cdb5c9d27)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:50.826606 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 195/375] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:05:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 05:05:50.827723 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb96d17-3c62-437d-9792-6c472723b619 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:50.829570 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 181/376] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Aug 31 05:05:50 2022] GET /identity/v3/groups/89d78f08484744609eb0db0cdb5c9d27 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:50.829919 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6858d090-e4a4-4e4f-92c7-63597bb4c9f6 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:50.835560 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb96d17-3c62-437d-9792-6c472723b619 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:50.965389 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6858d090-e4a4-4e4f-92c7-63597bb4c9f6 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:50.973295 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb96d17-3c62-437d-9792-6c472723b619 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:51.039042 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fbf7fd-dba2-473d-a5a5-f69273533de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:51.039694 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fbf7fd-dba2-473d-a5a5-f69273533de2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:51.039961 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fbf7fd-dba2-473d-a5a5-f69273533de2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:51.040232 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fbf7fd-dba2-473d-a5a5-f69273533de2 None admin] PATH_INFO: `/v3/projects/3e3bdb87df7f4087806a21456cb45a01` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:51.041844 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fbf7fd-dba2-473d-a5a5-f69273533de2 None admin] RBAC: Authorizing `identity:get_project(project_id=3e3bdb87df7f4087806a21456cb45a01)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:51.044963 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fbf7fd-dba2-473d-a5a5-f69273533de2 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:51.046966 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 196/377] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:05:50 2022] GET /identity/v3/projects/3e3bdb87df7f4087806a21456cb45a01 => generated 344 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:05:51.048572 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf645e8-7825-4454-a829-5b546d86bea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:51.049629 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf645e8-7825-4454-a829-5b546d86bea9 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:51.050150 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf645e8-7825-4454-a829-5b546d86bea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:51.050602 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf645e8-7825-4454-a829-5b546d86bea9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:51.051356 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf645e8-7825-4454-a829-5b546d86bea9 None admin] RBAC: Adding query filter params (group.id=89d78f08484744609eb0db0cdb5c9d27, role.id=a74bf01dd66f49f5988a197da269c822, scope.project.id=b08ea5fe820543b6936ffde78e21ea4f) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:51.051666 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf645e8-7825-4454-a829-5b546d86bea9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:51.053205 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf645e8-7825-4454-a829-5b546d86bea9 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:51.056196 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fbf7fd-dba2-473d-a5a5-f69273533de2 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:51.068535 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 182/378] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Aug 31 05:05:50 2022] GET /identity/v3/role_assignments?group.id=89d78f08484744609eb0db0cdb5c9d27&scope.project.id=b08ea5fe820543b6936ffde78e21ea4f&role.id=a74bf01dd66f49f5988a197da269c822 => generated 260 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:51.134335 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:51.190202 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fbf7fd-dba2-473d-a5a5-f69273533de2 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:51.234321 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58c9ab1-6c1b-48f1-86f4-45e3352c713b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:51.235034 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c9ab1-6c1b-48f1-86f4-45e3352c713b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:51.236473 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c9ab1-6c1b-48f1-86f4-45e3352c713b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:51.236743 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c9ab1-6c1b-48f1-86f4-45e3352c713b None admin] PATH_INFO: `/v3/groups/024e4c11d0144a6ba8107c6d169605ce` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:51.244604 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58c9ab1-6c1b-48f1-86f4-45e3352c713b None admin] RBAC: Authorizing `identity:get_group(group_id=024e4c11d0144a6ba8107c6d169605ce)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:51.246097 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58c9ab1-6c1b-48f1-86f4-45e3352c713b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:51.248111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 197/379] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Aug 31 05:05:51 2022] GET /identity/v3/groups/024e4c11d0144a6ba8107c6d169605ce => generated 229 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:51.253955 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf645e8-7825-4454-a829-5b546d86bea9 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:51.294844 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:51.390074 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf645e8-7825-4454-a829-5b546d86bea9 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:51.430906 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d0eae-a1d2-4c77-ba75-a4941a672a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:51.431548 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d0eae-a1d2-4c77-ba75-a4941a672a75 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:51.431893 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d0eae-a1d2-4c77-ba75-a4941a672a75 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:51.432169 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d0eae-a1d2-4c77-ba75-a4941a672a75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:51.433034 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02d0eae-a1d2-4c77-ba75-a4941a672a75 None admin] RBAC: Adding query filter params (group.id=024e4c11d0144a6ba8107c6d169605ce, role.id=c3d5589dd2864be9a0f26f5802d874cd, scope.project.id=3e3bdb87df7f4087806a21456cb45a01) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:51.433362 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02d0eae-a1d2-4c77-ba75-a4941a672a75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:51.435185 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02d0eae-a1d2-4c77-ba75-a4941a672a75 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:51.441836 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 183/380] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Aug 31 05:05:51 2022] GET /identity/v3/role_assignments?group.id=024e4c11d0144a6ba8107c6d169605ce&scope.project.id=3e3bdb87df7f4087806a21456cb45a01&role.id=c3d5589dd2864be9a0f26f5802d874cd => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:51.515073 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e2623f-857e-4103-bcd6-5711b01ab3db None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:51.515707 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e2623f-857e-4103-bcd6-5711b01ab3db None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:51.516204 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e2623f-857e-4103-bcd6-5711b01ab3db None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:51.518173 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 198/381] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:51.524202 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc43ab94-95fd-4717-bd2d-0de745fabeb0 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:51.524524 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc43ab94-95fd-4717-bd2d-0de745fabeb0 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:51.524833 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc43ab94-95fd-4717-bd2d-0de745fabeb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:51.556631 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-fc43ab94-95fd-4717-bd2d-0de745fabeb0 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:51.557847 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-fc43ab94-95fd-4717-bd2d-0de745fabeb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:51.632153 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b034ba78-307a-43e6-8a9e-6ff7bfae5a5f None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:51.632476 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b034ba78-307a-43e6-8a9e-6ff7bfae5a5f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:51.632688 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b034ba78-307a-43e6-8a9e-6ff7bfae5a5f None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:51.634330 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 199/382] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 05:05:51.641043 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f3b8d-539e-495c-90d2-bdac7d761865 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:51.641043 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f3b8d-539e-495c-90d2-bdac7d761865 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:51.641043 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f3b8d-539e-495c-90d2-bdac7d761865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:51.643497 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 184/383] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:51.670242 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-677f3b8d-539e-495c-90d2-bdac7d761865 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:51.671619 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-677f3b8d-539e-495c-90d2-bdac7d761865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:51.752438 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 200/384] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 05:05:52.031352 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc43ab94-95fd-4717-bd2d-0de745fabeb0 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:52.056235 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:52.072941 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677f3b8d-539e-495c-90d2-bdac7d761865 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:52.130362 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:52.184310 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc43ab94-95fd-4717-bd2d-0de745fabeb0 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:52.186281 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677f3b8d-539e-495c-90d2-bdac7d761865 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:52.222688 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e1afb-15b7-40d4-b0a2-6ded2048bbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:52.223465 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e1afb-15b7-40d4-b0a2-6ded2048bbe9 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:52.223791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e1afb-15b7-40d4-b0a2-6ded2048bbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:52.224099 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e1afb-15b7-40d4-b0a2-6ded2048bbe9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:52.235152 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1e1afb-15b7-40d4-b0a2-6ded2048bbe9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:52.240166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-bc1e1afb-15b7-40d4-b0a2-6ded2048bbe9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 05:05:52.240166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:52.240166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:52.240166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:52.240166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:52.240166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:52.240166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:52.240166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:52.240166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:52.240166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:52.240166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:52.240166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:52.240166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:52.240166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:52.240166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:52.240166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:52.241075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:52.241075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:52.241075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:52.241075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:52.241075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:52.241075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:52.241075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:52.241075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:52.241075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:52.241075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:52.241075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:52.241075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:52.241075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:52.241075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:52.241075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:52.241075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:52.241897 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:52.241897 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:52.241897 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:52.241897 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:52.241897 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:52.241897 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:52.241897 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:52.241897 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:52.241897 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:52.241897 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:52.241897 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 05:05:52.241897 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:05:52.241897 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 201/385] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:05:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:05:52.247718 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e1afb-15b7-40d4-b0a2-6ded2048bbe9 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:52.274411 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf00d3-21a4-4bfb-a6b7-e7c020a961d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:52.275065 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf00d3-21a4-4bfb-a6b7-e7c020a961d6 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:52.275343 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf00d3-21a4-4bfb-a6b7-e7c020a961d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:52.275615 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf00d3-21a4-4bfb-a6b7-e7c020a961d6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:52.294923 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf00d3-21a4-4bfb-a6b7-e7c020a961d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:52.299796 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-cfaf00d3-21a4-4bfb-a6b7-e7c020a961d6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 05:05:52.299796 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:52.299796 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:52.299796 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:52.299796 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:52.299796 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:52.299796 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:52.299796 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:52.299796 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:52.299796 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:52.299796 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:52.299796 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:52.299796 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:52.299796 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:52.299796 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:52.299796 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:52.300660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:52.300660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:52.300660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:52.300660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:52.300660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:52.300660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:52.300660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:52.300660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:52.300660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:52.300660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:52.300660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:52.300660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:52.300660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:52.300660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:52.300660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:52.300660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:52.301452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:52.301452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:52.301452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:52.301452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:52.301452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:52.301452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:52.301452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:52.301452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:52.301452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:52.301452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:52.301452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 05:05:52.301452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:05:52.306037 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 185/386] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 05:05:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:05:52.323886 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf00d3-21a4-4bfb-a6b7-e7c020a961d6 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:52.359986 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e1afb-15b7-40d4-b0a2-6ded2048bbe9 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:52.371870 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:52.398354 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd0e04b-571c-4c91-b60c-6fef8f797146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:52.398965 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0e04b-571c-4c91-b60c-6fef8f797146 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:52.399254 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0e04b-571c-4c91-b60c-6fef8f797146 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:52.399505 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0e04b-571c-4c91-b60c-6fef8f797146 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:52.400177 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd0e04b-571c-4c91-b60c-6fef8f797146 None admin] RBAC: Adding query filter params (name=member) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:52.400474 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd0e04b-571c-4c91-b60c-6fef8f797146 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:52.402020 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd0e04b-571c-4c91-b60c-6fef8f797146 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:52.411124 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 202/387] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 05:05:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:05:52.416816 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd0e04b-571c-4c91-b60c-6fef8f797146 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:52.519860 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:52.526826 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd0e04b-571c-4c91-b60c-6fef8f797146 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:52.564238 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b45c374-f0f9-4e71-a10d-31e086a11bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:52.564839 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45c374-f0f9-4e71-a10d-31e086a11bb2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:52.565124 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45c374-f0f9-4e71-a10d-31e086a11bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:52.565400 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45c374-f0f9-4e71-a10d-31e086a11bb2 None admin] PATH_INFO: `/v3/groups/89d78f08484744609eb0db0cdb5c9d27` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:52.566579 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b45c374-f0f9-4e71-a10d-31e086a11bb2 None admin] RBAC: Authorizing `identity:get_group(group_id=89d78f08484744609eb0db0cdb5c9d27)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:52.568295 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b45c374-f0f9-4e71-a10d-31e086a11bb2 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:52.570274 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 203/388] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Aug 31 05:05:52 2022] GET /identity/v3/groups/89d78f08484744609eb0db0cdb5c9d27 => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:05:52.576402 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b45c374-f0f9-4e71-a10d-31e086a11bb2 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:52.603964 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf00d3-21a4-4bfb-a6b7-e7c020a961d6 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:52.646450 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:52.680329 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef2d54e-c30d-480b-869f-565db674d49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:52.680944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2d54e-c30d-480b-869f-565db674d49e None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:52.681218 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2d54e-c30d-480b-869f-565db674d49e None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:52.681484 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2d54e-c30d-480b-869f-565db674d49e None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:52.682182 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef2d54e-c30d-480b-869f-565db674d49e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:52.682483 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef2d54e-c30d-480b-869f-565db674d49e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:52.683887 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef2d54e-c30d-480b-869f-565db674d49e None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:52.692739 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 186/389] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:05:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:05:52.696374 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b45c374-f0f9-4e71-a10d-31e086a11bb2 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:52.712290 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef2d54e-c30d-480b-869f-565db674d49e None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:52.751780 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845dc149-cd21-483b-a0a8-1a808827cf95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:52.752220 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845dc149-cd21-483b-a0a8-1a808827cf95 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:52.753860 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845dc149-cd21-483b-a0a8-1a808827cf95 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:52.754160 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845dc149-cd21-483b-a0a8-1a808827cf95 None admin] PATH_INFO: `/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:52.756757 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845dc149-cd21-483b-a0a8-1a808827cf95 None admin] RBAC: Authorizing `identity:get_project(project_id=f9d86697e3eb4fd6b8f82a013cfcb5ba)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:52.759714 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845dc149-cd21-483b-a0a8-1a808827cf95 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:52.764022 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 204/390] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:05:52 2022] GET /identity/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:52.770060 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845dc149-cd21-483b-a0a8-1a808827cf95 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:52.901452 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845dc149-cd21-483b-a0a8-1a808827cf95 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:52.931160 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:52.937455 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530fd45c-8561-406f-b505-5586c7dc1d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:52.938218 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530fd45c-8561-406f-b505-5586c7dc1d5b None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:52.938504 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530fd45c-8561-406f-b505-5586c7dc1d5b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:52.938780 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530fd45c-8561-406f-b505-5586c7dc1d5b None admin] PATH_INFO: `/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba/groups/89d78f08484744609eb0db0cdb5c9d27/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:52.941177 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530fd45c-8561-406f-b505-5586c7dc1d5b None admin] RBAC: Authorizing `identity:create_grant(project_id=f9d86697e3eb4fd6b8f82a013cfcb5ba, group_id=89d78f08484744609eb0db0cdb5c9d27, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:52.942814 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530fd45c-8561-406f-b505-5586c7dc1d5b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:52.949717 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef2d54e-c30d-480b-869f-565db674d49e None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:52.962893 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 205/391] 10.222.0.36 () {58 vars in 1442 bytes} [Wed Aug 31 05:05:52 2022] PUT /identity/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba/groups/89d78f08484744609eb0db0cdb5c9d27/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:05:53.032013 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5108be51-afbd-4498-a5d0-1b20038610e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:53.032684 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5108be51-afbd-4498-a5d0-1b20038610e6 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:53.032960 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5108be51-afbd-4498-a5d0-1b20038610e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:53.033228 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5108be51-afbd-4498-a5d0-1b20038610e6 None admin] PATH_INFO: `/v3/groups/89d78f08484744609eb0db0cdb5c9d27` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:53.034460 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5108be51-afbd-4498-a5d0-1b20038610e6 None admin] RBAC: Authorizing `identity:get_group(group_id=89d78f08484744609eb0db0cdb5c9d27)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:53.035971 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5108be51-afbd-4498-a5d0-1b20038610e6 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:53.037666 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 187/392] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Aug 31 05:05:52 2022] GET /identity/v3/groups/89d78f08484744609eb0db0cdb5c9d27 => generated 226 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:53.043771 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530fd45c-8561-406f-b505-5586c7dc1d5b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:53.170369 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530fd45c-8561-406f-b505-5586c7dc1d5b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:53.191601 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf85a96-c3b7-4b38-b207-2d554f9d79f3 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:53.191897 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf85a96-c3b7-4b38-b207-2d554f9d79f3 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:53.192154 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf85a96-c3b7-4b38-b207-2d554f9d79f3 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:53.194154 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 188/393] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:53.199375 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f906b9e-cc07-4f77-8260-f40eb50c9e05 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:53.199660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f906b9e-cc07-4f77-8260-f40eb50c9e05 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:53.199937 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f906b9e-cc07-4f77-8260-f40eb50c9e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:53.210382 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3307d45b-b8a4-47bf-b238-8456ceb40ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:53.211497 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307d45b-b8a4-47bf-b238-8456ceb40ed3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:53.212011 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307d45b-b8a4-47bf-b238-8456ceb40ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:53.212461 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307d45b-b8a4-47bf-b238-8456ceb40ed3 None admin] PATH_INFO: `/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:53.213838 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3307d45b-b8a4-47bf-b238-8456ceb40ed3 None admin] RBAC: Authorizing `identity:get_project(project_id=f9d86697e3eb4fd6b8f82a013cfcb5ba)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:53.215618 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3307d45b-b8a4-47bf-b238-8456ceb40ed3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:53.218274 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 206/394] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:05:53 2022] GET /identity/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:53.225456 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3307d45b-b8a4-47bf-b238-8456ceb40ed3 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:53.238267 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-2f906b9e-cc07-4f77-8260-f40eb50c9e05 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:53.239322 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-2f906b9e-cc07-4f77-8260-f40eb50c9e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:53.242313 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:53.307119 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:53.357144 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3307d45b-b8a4-47bf-b238-8456ceb40ed3 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:53.363651 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 189/395] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:53.392075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a6b6ae-b3e2-4bef-9173-3603412b2958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:53.392742 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6b6ae-b3e2-4bef-9173-3603412b2958 None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:53.393022 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6b6ae-b3e2-4bef-9173-3603412b2958 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:53.393292 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6b6ae-b3e2-4bef-9173-3603412b2958 None admin] PATH_INFO: `/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba/groups/89d78f08484744609eb0db0cdb5c9d27/roles/a74bf01dd66f49f5988a197da269c822` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:53.395526 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a6b6ae-b3e2-4bef-9173-3603412b2958 None admin] RBAC: Authorizing `identity:create_grant(project_id=f9d86697e3eb4fd6b8f82a013cfcb5ba, group_id=89d78f08484744609eb0db0cdb5c9d27, role_id=a74bf01dd66f49f5988a197da269c822)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:53.396978 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a6b6ae-b3e2-4bef-9173-3603412b2958 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:53.411522 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 207/396] 10.222.0.36 () {58 vars in 1442 bytes} [Wed Aug 31 05:05:53 2022] PUT /identity/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba/groups/89d78f08484744609eb0db0cdb5c9d27/roles/a74bf01dd66f49f5988a197da269c822 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:05:53.712778 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f906b9e-cc07-4f77-8260-f40eb50c9e05 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:53.714180 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f78539-bd6a-475c-abb9-949676c5121a None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:53.714473 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f78539-bd6a-475c-abb9-949676c5121a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:53.714750 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f78539-bd6a-475c-abb9-949676c5121a None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:53.716754 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 208/397] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:53.721592 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229d7dd-bae3-4445-b417-49efd21b58a9 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:53.721853 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229d7dd-bae3-4445-b417-49efd21b58a9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:53.722173 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229d7dd-bae3-4445-b417-49efd21b58a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:53.741207 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-1229d7dd-bae3-4445-b417-49efd21b58a9 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:53.742525 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1229d7dd-bae3-4445-b417-49efd21b58a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:53.851764 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 209/398] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 05:05:53.879561 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f906b9e-cc07-4f77-8260-f40eb50c9e05 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:53.916956 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f414d2-93d7-494e-a717-03b5a656eab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:53.917517 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f414d2-93d7-494e-a717-03b5a656eab4 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:53.917792 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f414d2-93d7-494e-a717-03b5a656eab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:53.918088 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f414d2-93d7-494e-a717-03b5a656eab4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:53.926037 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:53.930160 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f414d2-93d7-494e-a717-03b5a656eab4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:53.935044 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-d6f414d2-93d7-494e-a717-03b5a656eab4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 05:05:53.935044 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:53.935044 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:53.935044 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:53.935044 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:53.935044 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:53.935044 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:53.935044 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:53.935044 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:53.935044 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:53.935044 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:53.935044 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:53.935044 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:53.935044 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:53.935044 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:53.935044 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:53.935966 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:53.935966 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:53.935966 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:53.935966 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:53.935966 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:53.935966 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:53.935966 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:53.935966 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:53.935966 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:53.935966 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:53.935966 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:53.935966 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:53.935966 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:53.935966 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:53.935966 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:53.935966 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:53.936849 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:53.936849 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:53.936849 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:53.936849 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:53.936849 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:53.936849 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:53.936849 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:53.936849 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:53.936849 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:53.936849 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:53.936849 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 05:05:53.936849 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:05:53.936849 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 190/399] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:05:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:05:53.942145 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1229d7dd-bae3-4445-b417-49efd21b58a9 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:53.983634 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:54.072920 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1229d7dd-bae3-4445-b417-49efd21b58a9 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:54.107171 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f98fb8-c4ee-420d-9e3a-187c6d7ce043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:54.107786 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f98fb8-c4ee-420d-9e3a-187c6d7ce043 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:54.108146 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f98fb8-c4ee-420d-9e3a-187c6d7ce043 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:54.108382 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f98fb8-c4ee-420d-9e3a-187c6d7ce043 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:54.109109 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f98fb8-c4ee-420d-9e3a-187c6d7ce043 None admin] RBAC: Adding query filter params (name=member) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:54.109416 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f98fb8-c4ee-420d-9e3a-187c6d7ce043 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:54.110901 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f98fb8-c4ee-420d-9e3a-187c6d7ce043 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:54.119893 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e9dfd-5525-4557-9d7f-95b86bb3dd91 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:54.120188 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e9dfd-5525-4557-9d7f-95b86bb3dd91 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:54.120450 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e9dfd-5525-4557-9d7f-95b86bb3dd91 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:54.122034 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 191/401] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:54.122792 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 210/401] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 05:05:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:54.126181 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e9dfd-5525-4557-9d7f-95b86bb3dd91 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:54.126919 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82101e-1cd7-48d9-bc92-27d0d7712c9a None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:54.127209 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82101e-1cd7-48d9-bc92-27d0d7712c9a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:54.127486 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82101e-1cd7-48d9-bc92-27d0d7712c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:54.147035 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ab82101e-1cd7-48d9-bc92-27d0d7712c9a None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:54.148224 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ab82101e-1cd7-48d9-bc92-27d0d7712c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:54.261791 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e9dfd-5525-4557-9d7f-95b86bb3dd91 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:54.267615 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 211/402] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 05:05:54.323826 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ad12fa-bedb-493c-bfbb-4bf603114741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:54.324464 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad12fa-bedb-493c-bfbb-4bf603114741 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:54.324741 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad12fa-bedb-493c-bfbb-4bf603114741 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:54.325010 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad12fa-bedb-493c-bfbb-4bf603114741 None admin] PATH_INFO: `/v3/groups/89d78f08484744609eb0db0cdb5c9d27` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:54.328390 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ad12fa-bedb-493c-bfbb-4bf603114741 None admin] RBAC: Authorizing `identity:get_group(group_id=89d78f08484744609eb0db0cdb5c9d27)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:54.329269 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab82101e-1cd7-48d9-bc92-27d0d7712c9a None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:54.329868 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ad12fa-bedb-493c-bfbb-4bf603114741 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:54.331495 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 192/403] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Aug 31 05:05:54 2022] GET /identity/v3/groups/89d78f08484744609eb0db0cdb5c9d27 => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:54.338348 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ad12fa-bedb-493c-bfbb-4bf603114741 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:54.360614 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:54.443640 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:54.501439 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ad12fa-bedb-493c-bfbb-4bf603114741 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:54.506225 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab82101e-1cd7-48d9-bc92-27d0d7712c9a None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:54.547645 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ad760-d14a-4a2a-8682-4db547e66635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:54.548197 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ad760-d14a-4a2a-8682-4db547e66635 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:54.548473 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ad760-d14a-4a2a-8682-4db547e66635 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:54.548744 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ad760-d14a-4a2a-8682-4db547e66635 None admin] PATH_INFO: `/v3/projects/b08ea5fe820543b6936ffde78e21ea4f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:54.549859 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342ad760-d14a-4a2a-8682-4db547e66635 None admin] RBAC: Authorizing `identity:get_project(project_id=b08ea5fe820543b6936ffde78e21ea4f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:54.551332 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342ad760-d14a-4a2a-8682-4db547e66635 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:54.552970 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 193/404] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:05:54 2022] GET /identity/v3/projects/b08ea5fe820543b6936ffde78e21ea4f => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:05:54.559403 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ad760-d14a-4a2a-8682-4db547e66635 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:54.578354 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de38894-5d2b-4437-ad2c-3f44839ee215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:54.579022 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de38894-5d2b-4437-ad2c-3f44839ee215 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:54.579270 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de38894-5d2b-4437-ad2c-3f44839ee215 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:54.579532 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de38894-5d2b-4437-ad2c-3f44839ee215 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:54.592533 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de38894-5d2b-4437-ad2c-3f44839ee215 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:54.599598 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-7de38894-5d2b-4437-ad2c-3f44839ee215 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 05:05:54.599598 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:54.599598 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:54.599598 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:54.599598 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:54.599598 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:54.599598 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:54.599598 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:54.599598 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:54.599598 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:54.599598 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:54.599598 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:54.599598 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:54.599598 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:54.599598 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:54.599598 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:54.600516 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:54.600516 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:54.600516 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:54.600516 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:54.600516 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:54.600516 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:54.600516 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:54.600516 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:54.600516 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:54.600516 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:54.600516 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:54.600516 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:54.600516 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:54.600516 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:54.600516 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:54.600516 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:54.601327 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:54.601327 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:54.601327 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:54.601327 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:54.601327 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:54.601327 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:54.601327 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:54.601327 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:54.601327 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:54.601327 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:54.601327 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 05:05:54.601327 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:05:54.601327 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 212/405] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 05:05:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:05:54.602034 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de38894-5d2b-4437-ad2c-3f44839ee215 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:54.675810 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ad760-d14a-4a2a-8682-4db547e66635 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:54.700002 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:54.713452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b56472-8eb1-4beb-a1a2-2ac273c44431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:54.714150 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b56472-8eb1-4beb-a1a2-2ac273c44431 None admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:54.714431 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b56472-8eb1-4beb-a1a2-2ac273c44431 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:54.714700 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b56472-8eb1-4beb-a1a2-2ac273c44431 None admin] PATH_INFO: `/v3/projects/b08ea5fe820543b6936ffde78e21ea4f/groups/89d78f08484744609eb0db0cdb5c9d27/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:54.716913 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b56472-8eb1-4beb-a1a2-2ac273c44431 None admin] RBAC: Authorizing `identity:create_grant(project_id=b08ea5fe820543b6936ffde78e21ea4f, group_id=89d78f08484744609eb0db0cdb5c9d27, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:54.718385 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b56472-8eb1-4beb-a1a2-2ac273c44431 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:54.721887 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:54.726092 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de38894-5d2b-4437-ad2c-3f44839ee215 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:54.737387 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 194/406] 10.222.0.36 () {58 vars in 1442 bytes} [Wed Aug 31 05:05:54 2022] PUT /identity/v3/projects/b08ea5fe820543b6936ffde78e21ea4f/groups/89d78f08484744609eb0db0cdb5c9d27/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:05:54.744743 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b56472-8eb1-4beb-a1a2-2ac273c44431 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:54.767008 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e97d6e-fd90-44f5-a3ab-1c05188ba820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:54.768114 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e97d6e-fd90-44f5-a3ab-1c05188ba820 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:54.768633 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e97d6e-fd90-44f5-a3ab-1c05188ba820 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:54.769105 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e97d6e-fd90-44f5-a3ab-1c05188ba820 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:54.782389 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e97d6e-fd90-44f5-a3ab-1c05188ba820 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:54.787887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-36e97d6e-fd90-44f5-a3ab-1c05188ba820 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:05:54.787887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:54.787887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:54.787887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:54.787887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:54.787887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:54.787887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:54.787887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:54.787887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:54.787887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:54.787887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:54.787887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:54.787887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:54.787887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:54.787887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:54.787887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:54.788860 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:54.788860 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:54.788860 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:54.788860 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:54.788860 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:54.788860 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:54.788860 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:54.788860 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:54.788860 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:54.788860 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:54.788860 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:54.788860 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:54.788860 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:54.788860 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:54.788860 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:54.788860 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:54.789692 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:54.789692 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:54.789692 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:54.789692 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:54.789692 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:54.789692 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:54.789692 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:54.789692 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:54.789692 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:54.789692 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:54.789692 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:05:54.789692 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:05:54.792094 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 213/407] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 05:05:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:05:54.799530 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e97d6e-fd90-44f5-a3ab-1c05188ba820 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:54.912394 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e97d6e-fd90-44f5-a3ab-1c05188ba820 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:54.949588 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9df968-5300-4bcd-8197-5cd1182754a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:54.950642 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9df968-5300-4bcd-8197-5cd1182754a5 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:54.951148 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9df968-5300-4bcd-8197-5cd1182754a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:54.951657 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9df968-5300-4bcd-8197-5cd1182754a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:54.952609 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9df968-5300-4bcd-8197-5cd1182754a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:54.953115 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9df968-5300-4bcd-8197-5cd1182754a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:54.954806 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9df968-5300-4bcd-8197-5cd1182754a5 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:54.957146 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b56472-8eb1-4beb-a1a2-2ac273c44431 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:54.961328 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:54.965959 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 214/408] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:05:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:05:54.971946 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9df968-5300-4bcd-8197-5cd1182754a5 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:55.032694 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d19f59-7307-4433-b48a-f915b8c892b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:55.033339 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d19f59-7307-4433-b48a-f915b8c892b5 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:55.033701 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d19f59-7307-4433-b48a-f915b8c892b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:55.033878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d19f59-7307-4433-b48a-f915b8c892b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:55.034586 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d19f59-7307-4433-b48a-f915b8c892b5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:55.034892 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d19f59-7307-4433-b48a-f915b8c892b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:55.036294 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d19f59-7307-4433-b48a-f915b8c892b5 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:55.048075 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 195/409] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:05:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:55.058872 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d19f59-7307-4433-b48a-f915b8c892b5 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:55.094101 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9df968-5300-4bcd-8197-5cd1182754a5 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:55.149304 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a328115-bf40-4dcf-b23b-7e6da1bfffc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:55.149941 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a328115-bf40-4dcf-b23b-7e6da1bfffc2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:55.150247 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a328115-bf40-4dcf-b23b-7e6da1bfffc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:55.150512 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a328115-bf40-4dcf-b23b-7e6da1bfffc2 None admin] PATH_INFO: `/v3/groups/024e4c11d0144a6ba8107c6d169605ce` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:55.153540 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:55.155643 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a328115-bf40-4dcf-b23b-7e6da1bfffc2 None admin] RBAC: Authorizing `identity:get_group(group_id=024e4c11d0144a6ba8107c6d169605ce)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:55.159446 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a328115-bf40-4dcf-b23b-7e6da1bfffc2 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:55.161259 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 215/410] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Aug 31 05:05:54 2022] GET /identity/v3/groups/024e4c11d0144a6ba8107c6d169605ce => generated 229 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:55.166814 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a328115-bf40-4dcf-b23b-7e6da1bfffc2 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:55.231315 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d19f59-7307-4433-b48a-f915b8c892b5 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:55.280758 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cef51d-3d6e-464e-ab37-bbf7d2b7c4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:55.281407 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cef51d-3d6e-464e-ab37-bbf7d2b7c4b3 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:55.281735 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cef51d-3d6e-464e-ab37-bbf7d2b7c4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:55.282294 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cef51d-3d6e-464e-ab37-bbf7d2b7c4b3 None admin] PATH_INFO: `/v3/groups/89d78f08484744609eb0db0cdb5c9d27` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:55.285136 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cef51d-3d6e-464e-ab37-bbf7d2b7c4b3 None admin] RBAC: Authorizing `identity:get_group(group_id=89d78f08484744609eb0db0cdb5c9d27)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:55.287464 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cef51d-3d6e-464e-ab37-bbf7d2b7c4b3 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:55.289726 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 196/411] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Aug 31 05:05:55 2022] GET /identity/v3/groups/89d78f08484744609eb0db0cdb5c9d27 => generated 226 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:55.295064 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cef51d-3d6e-464e-ab37-bbf7d2b7c4b3 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:55.297285 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:55.337954 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a328115-bf40-4dcf-b23b-7e6da1bfffc2 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:55.399655 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2366a66b-7b29-45c4-9e3c-e7dd5d0bf3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:55.400686 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366a66b-7b29-45c4-9e3c-e7dd5d0bf3da None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:55.401296 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366a66b-7b29-45c4-9e3c-e7dd5d0bf3da None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:55.401759 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366a66b-7b29-45c4-9e3c-e7dd5d0bf3da None admin] PATH_INFO: `/v3/projects/3e3bdb87df7f4087806a21456cb45a01` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:55.403125 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2366a66b-7b29-45c4-9e3c-e7dd5d0bf3da None admin] RBAC: Authorizing `identity:get_project(project_id=3e3bdb87df7f4087806a21456cb45a01)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:55.404918 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2366a66b-7b29-45c4-9e3c-e7dd5d0bf3da None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:55.414421 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 216/412] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:05:55 2022] GET /identity/v3/projects/3e3bdb87df7f4087806a21456cb45a01 => generated 344 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:55.415529 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2366a66b-7b29-45c4-9e3c-e7dd5d0bf3da None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:55.441735 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cef51d-3d6e-464e-ab37-bbf7d2b7c4b3 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:55.469807 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:55.480981 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416da839-6a8d-4a13-9639-439996f381a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:55.481620 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416da839-6a8d-4a13-9639-439996f381a9 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:55.481900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416da839-6a8d-4a13-9639-439996f381a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:55.482205 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416da839-6a8d-4a13-9639-439996f381a9 None admin] PATH_INFO: `/v3/projects/b08ea5fe820543b6936ffde78e21ea4f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:55.483482 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416da839-6a8d-4a13-9639-439996f381a9 None admin] RBAC: Authorizing `identity:get_project(project_id=b08ea5fe820543b6936ffde78e21ea4f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:55.485382 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416da839-6a8d-4a13-9639-439996f381a9 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:55.487327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 197/413] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:05:55 2022] GET /identity/v3/projects/b08ea5fe820543b6936ffde78e21ea4f => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:55.488754 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78d44a-8856-491f-bcd9-178b0bb24466 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:55.489036 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78d44a-8856-491f-bcd9-178b0bb24466 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:55.489304 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78d44a-8856-491f-bcd9-178b0bb24466 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:55.495039 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 198/414] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:55.495653 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af78d44a-8856-491f-bcd9-178b0bb24466 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:55.524178 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2366a66b-7b29-45c4-9e3c-e7dd5d0bf3da None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:55.558645 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d0b029-c3d9-4f0d-8653-6985b23c9f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:55.559365 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d0b029-c3d9-4f0d-8653-6985b23c9f4d None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:55.559646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d0b029-c3d9-4f0d-8653-6985b23c9f4d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:55.559916 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d0b029-c3d9-4f0d-8653-6985b23c9f4d None admin] PATH_INFO: `/v3/projects/3e3bdb87df7f4087806a21456cb45a01/groups/024e4c11d0144a6ba8107c6d169605ce/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:55.561843 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d0b029-c3d9-4f0d-8653-6985b23c9f4d None admin] RBAC: Authorizing `identity:create_grant(project_id=3e3bdb87df7f4087806a21456cb45a01, group_id=024e4c11d0144a6ba8107c6d169605ce, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:55.563306 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d0b029-c3d9-4f0d-8653-6985b23c9f4d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:55.566371 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:55.582103 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 217/415] 10.222.0.36 () {58 vars in 1442 bytes} [Wed Aug 31 05:05:55 2022] PUT /identity/v3/projects/3e3bdb87df7f4087806a21456cb45a01/groups/024e4c11d0144a6ba8107c6d169605ce/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:05:55.586347 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d273e8eb-e5ed-447d-a613-fd18ec9766cd None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:55.587036 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d273e8eb-e5ed-447d-a613-fd18ec9766cd None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:55.587652 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d273e8eb-e5ed-447d-a613-fd18ec9766cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:55.609125 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d273e8eb-e5ed-447d-a613-fd18ec9766cd None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:55.610731 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d273e8eb-e5ed-447d-a613-fd18ec9766cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:55.650047 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af78d44a-8856-491f-bcd9-178b0bb24466 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:55.693291 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0597cbf-b0f6-4417-832c-0d819df5a359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:55.693993 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0597cbf-b0f6-4417-832c-0d819df5a359 None admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:55.694301 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0597cbf-b0f6-4417-832c-0d819df5a359 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:55.694574 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0597cbf-b0f6-4417-832c-0d819df5a359 None admin] PATH_INFO: `/v3/projects/b08ea5fe820543b6936ffde78e21ea4f/groups/89d78f08484744609eb0db0cdb5c9d27/roles/a74bf01dd66f49f5988a197da269c822` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:55.696831 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0597cbf-b0f6-4417-832c-0d819df5a359 None admin] RBAC: Authorizing `identity:create_grant(project_id=b08ea5fe820543b6936ffde78e21ea4f, group_id=89d78f08484744609eb0db0cdb5c9d27, role_id=a74bf01dd66f49f5988a197da269c822)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:55.698422 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0597cbf-b0f6-4417-832c-0d819df5a359 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:55.698731 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 218/416] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:55.706527 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb3cdd-0502-425c-9e76-326216e717d8 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:55.706824 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb3cdd-0502-425c-9e76-326216e717d8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:55.707155 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb3cdd-0502-425c-9e76-326216e717d8 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:55.713231 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 219/417] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:55 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:55.715150 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965be543-03ac-4d3f-84d2-3a35a676c585 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:55.715438 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965be543-03ac-4d3f-84d2-3a35a676c585 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:55.715968 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965be543-03ac-4d3f-84d2-3a35a676c585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:55.725461 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 199/418] 10.222.0.36 () {58 vars in 1442 bytes} [Wed Aug 31 05:05:55 2022] PUT /identity/v3/projects/b08ea5fe820543b6936ffde78e21ea4f/groups/89d78f08484744609eb0db0cdb5c9d27/roles/a74bf01dd66f49f5988a197da269c822 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:05:55.744711 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-965be543-03ac-4d3f-84d2-3a35a676c585 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:55.746111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-965be543-03ac-4d3f-84d2-3a35a676c585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:55.829257 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 220/419] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:56.009581 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0597cbf-b0f6-4417-832c-0d819df5a359 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:56.083734 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:56.131150 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965be543-03ac-4d3f-84d2-3a35a676c585 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:56.137103 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0597cbf-b0f6-4417-832c-0d819df5a359 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:56.196581 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a235560-1ab6-455d-a904-2f998ad98378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:56.197638 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a235560-1ab6-455d-a904-2f998ad98378 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:56.198171 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a235560-1ab6-455d-a904-2f998ad98378 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:56.198645 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a235560-1ab6-455d-a904-2f998ad98378 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:56.217795 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:56.223364 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a235560-1ab6-455d-a904-2f998ad98378 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:56.229175 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-6a235560-1ab6-455d-a904-2f998ad98378 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 05:05:56.229175 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:56.229175 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:56.229175 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:56.229175 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:56.229175 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:56.229175 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:56.229175 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:56.229175 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:56.229175 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:56.229175 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:56.229175 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:56.229175 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:56.229175 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:56.229175 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:56.229175 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:56.230161 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:56.230161 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:56.230161 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:56.230161 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:56.230161 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:56.230161 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:56.230161 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:56.230161 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:56.230161 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:56.230161 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:56.230161 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:56.230161 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:56.230161 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:56.230161 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:56.230161 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:56.230161 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:56.230959 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:56.230959 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:56.230959 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:56.230959 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:56.230959 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:56.230959 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:56.230959 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:56.230959 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:56.230959 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:56.230959 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:56.230959 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 05:05:56.230959 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:05:56.239579 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 200/420] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:05:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:05:56.246251 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a235560-1ab6-455d-a904-2f998ad98378 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:56.298896 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965be543-03ac-4d3f-84d2-3a35a676c585 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:56.355655 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e626a7-d6cf-4a03-8a3f-0fe060ba936c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:56.356297 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e626a7-d6cf-4a03-8a3f-0fe060ba936c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:56.356577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e626a7-d6cf-4a03-8a3f-0fe060ba936c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:56.356842 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e626a7-d6cf-4a03-8a3f-0fe060ba936c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:56.379665 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e626a7-d6cf-4a03-8a3f-0fe060ba936c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:56.387996 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-b5e626a7-d6cf-4a03-8a3f-0fe060ba936c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 05:05:56.387996 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:56.387996 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:56.387996 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:56.387996 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:56.387996 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:56.387996 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:56.387996 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:56.387996 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:56.387996 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:56.387996 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:56.387996 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:56.387996 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:56.387996 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:56.387996 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:56.387996 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:56.388909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:56.388909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:56.388909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:56.388909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:56.388909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:56.388909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:56.388909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:56.388909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:56.388909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:56.388909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:56.388909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:56.388909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:56.388909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:56.388909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:56.388909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:56.388909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:56.389780 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:56.389780 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:56.389780 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:56.389780 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:56.389780 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:56.389780 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:56.389780 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:56.389780 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:56.389780 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:56.389780 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:56.389780 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 05:05:56.389780 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:05:56.395832 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 221/421] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 05:05:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:05:56.402979 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e626a7-d6cf-4a03-8a3f-0fe060ba936c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:56.409506 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a235560-1ab6-455d-a904-2f998ad98378 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:56.425841 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:56.453148 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e56614a-490a-405e-9ae7-a8da4dd08e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:56.453765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56614a-490a-405e-9ae7-a8da4dd08e66 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:56.454066 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56614a-490a-405e-9ae7-a8da4dd08e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:56.454341 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56614a-490a-405e-9ae7-a8da4dd08e66 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:56.455009 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e56614a-490a-405e-9ae7-a8da4dd08e66 None admin] RBAC: Adding query filter params (name=member) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:56.455310 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e56614a-490a-405e-9ae7-a8da4dd08e66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:56.456696 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e56614a-490a-405e-9ae7-a8da4dd08e66 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:56.484690 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 201/422] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 05:05:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:05:56.499453 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e56614a-490a-405e-9ae7-a8da4dd08e66 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:56.611209 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e626a7-d6cf-4a03-8a3f-0fe060ba936c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:56.663360 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:56.668933 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e56614a-490a-405e-9ae7-a8da4dd08e66 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:56.697130 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f69c78-cba2-4faa-a7d1-af5536515149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:56.697792 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f69c78-cba2-4faa-a7d1-af5536515149 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:56.698099 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f69c78-cba2-4faa-a7d1-af5536515149 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:56.698371 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f69c78-cba2-4faa-a7d1-af5536515149 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:56.699057 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f69c78-cba2-4faa-a7d1-af5536515149 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:56.699336 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f69c78-cba2-4faa-a7d1-af5536515149 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:56.700722 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f69c78-cba2-4faa-a7d1-af5536515149 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:56.713836 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3eb08e-1793-4769-ac8a-5fe2f3069774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:56.717162 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3eb08e-1793-4769-ac8a-5fe2f3069774 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:56.717511 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3eb08e-1793-4769-ac8a-5fe2f3069774 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:56.717784 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3eb08e-1793-4769-ac8a-5fe2f3069774 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:56.723207 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 222/423] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:05:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:56.728912 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f69c78-cba2-4faa-a7d1-af5536515149 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:56.787343 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:56.806645 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3eb08e-1793-4769-ac8a-5fe2f3069774 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:56.808344 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3eb08e-1793-4769-ac8a-5fe2f3069774 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:56.916475 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 202/424] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:05:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 05:05:56.922230 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3eb08e-1793-4769-ac8a-5fe2f3069774 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:57.004036 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f69c78-cba2-4faa-a7d1-af5536515149 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:57.004693 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:57.030607 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3eb08e-1793-4769-ac8a-5fe2f3069774 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:57.066548 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8cdfd-3756-497f-8463-728e92de5057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:57.067266 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8cdfd-3756-497f-8463-728e92de5057 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:57.067644 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8cdfd-3756-497f-8463-728e92de5057 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:57.067945 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8cdfd-3756-497f-8463-728e92de5057 None admin] PATH_INFO: `/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:57.069097 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8cdfd-3756-497f-8463-728e92de5057 None admin] RBAC: Authorizing `identity:get_project(project_id=f9d86697e3eb4fd6b8f82a013cfcb5ba)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:57.070614 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8cdfd-3756-497f-8463-728e92de5057 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:57.072655 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 203/425] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:05:56 2022] GET /identity/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:57.078190 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8cdfd-3756-497f-8463-728e92de5057 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:57.126746 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbb524e-b4cc-4e3f-8d21-9040d666eb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:57.127378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb524e-b4cc-4e3f-8d21-9040d666eb58 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:57.127655 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb524e-b4cc-4e3f-8d21-9040d666eb58 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:57.127921 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb524e-b4cc-4e3f-8d21-9040d666eb58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:57.184805 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8cdfd-3756-497f-8463-728e92de5057 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:57.207319 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:57.228912 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9794846-c92f-4073-b07a-8350bf255844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:57.229546 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9794846-c92f-4073-b07a-8350bf255844 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:57.229822 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9794846-c92f-4073-b07a-8350bf255844 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:57.230111 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9794846-c92f-4073-b07a-8350bf255844 None admin] PATH_INFO: `/v3/groups/89d78f08484744609eb0db0cdb5c9d27` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:57.231249 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9794846-c92f-4073-b07a-8350bf255844 None admin] RBAC: Authorizing `identity:get_group(group_id=89d78f08484744609eb0db0cdb5c9d27)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:57.232694 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9794846-c92f-4073-b07a-8350bf255844 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:57.234689 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 204/426] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Aug 31 05:05:57 2022] GET /identity/v3/groups/89d78f08484744609eb0db0cdb5c9d27 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:05:57.235833 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f4d5e-89b7-474b-94ab-a78437910b10 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:57.236111 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f4d5e-89b7-474b-94ab-a78437910b10 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:57.236382 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f4d5e-89b7-474b-94ab-a78437910b10 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:57.238979 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbb524e-b4cc-4e3f-8d21-9040d666eb58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:57.240535 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbb524e-b4cc-4e3f-8d21-9040d666eb58 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:57.242532 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 205/427] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:57.243315 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f4d5e-89b7-474b-94ab-a78437910b10 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:57.259493 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:57.355263 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f4d5e-89b7-474b-94ab-a78437910b10 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:57.378651 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 223/428] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:05:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 645 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 05:05:57.380196 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95826568-145a-44f3-826c-e907c5fdea85 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:57.380484 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95826568-145a-44f3-826c-e907c5fdea85 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:57.380755 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95826568-145a-44f3-826c-e907c5fdea85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:57.391612 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599ebdc-c127-483c-8d03-161fa59bacdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:57.392221 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599ebdc-c127-483c-8d03-161fa59bacdf None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:57.392498 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599ebdc-c127-483c-8d03-161fa59bacdf None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:57.392812 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599ebdc-c127-483c-8d03-161fa59bacdf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:57.393555 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0599ebdc-c127-483c-8d03-161fa59bacdf None admin] RBAC: Adding query filter params (group.id=89d78f08484744609eb0db0cdb5c9d27, role.id=e3235eda7c654e2abf96ed6a494f5152, scope.project.id=f9d86697e3eb4fd6b8f82a013cfcb5ba) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:57.393863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0599ebdc-c127-483c-8d03-161fa59bacdf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:57.395385 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0599ebdc-c127-483c-8d03-161fa59bacdf None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:57.400117 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-95826568-145a-44f3-826c-e907c5fdea85 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:57.401355 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-95826568-145a-44f3-826c-e907c5fdea85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:57.401639 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 206/429] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Aug 31 05:05:57 2022] GET /identity/v3/role_assignments?group.id=89d78f08484744609eb0db0cdb5c9d27&scope.project.id=f9d86697e3eb4fd6b8f82a013cfcb5ba&role.id=e3235eda7c654e2abf96ed6a494f5152 => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:57.407714 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599ebdc-c127-483c-8d03-161fa59bacdf None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:57.476263 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 224/430] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:05:57.535684 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:57.570330 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599ebdc-c127-483c-8d03-161fa59bacdf None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:57.617393 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa51c75-ddde-417e-97c8-6d7627c62c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:57.618089 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa51c75-ddde-417e-97c8-6d7627c62c84 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:57.618640 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa51c75-ddde-417e-97c8-6d7627c62c84 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:57.618916 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa51c75-ddde-417e-97c8-6d7627c62c84 None admin] PATH_INFO: `/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:57.620037 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa51c75-ddde-417e-97c8-6d7627c62c84 None admin] RBAC: Authorizing `identity:get_project(project_id=f9d86697e3eb4fd6b8f82a013cfcb5ba)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:57.621479 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa51c75-ddde-417e-97c8-6d7627c62c84 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:57.623146 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 207/431] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:05:57 2022] GET /identity/v3/projects/f9d86697e3eb4fd6b8f82a013cfcb5ba => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:57.628994 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95826568-145a-44f3-826c-e907c5fdea85 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:57.641318 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:57.732792 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95826568-145a-44f3-826c-e907c5fdea85 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:57.768499 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e149e2bf-1f4a-46c0-822f-7ef4a5ccb744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:57.769129 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149e2bf-1f4a-46c0-822f-7ef4a5ccb744 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:57.769402 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149e2bf-1f4a-46c0-822f-7ef4a5ccb744 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:57.769671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149e2bf-1f4a-46c0-822f-7ef4a5ccb744 None admin] PATH_INFO: `/v3/groups/89d78f08484744609eb0db0cdb5c9d27` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:57.770803 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e149e2bf-1f4a-46c0-822f-7ef4a5ccb744 None admin] RBAC: Authorizing `identity:get_group(group_id=89d78f08484744609eb0db0cdb5c9d27)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:57.772243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e149e2bf-1f4a-46c0-822f-7ef4a5ccb744 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:57.778112 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 225/432] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Aug 31 05:05:57 2022] GET /identity/v3/groups/89d78f08484744609eb0db0cdb5c9d27 => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:57.780036 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa51c75-ddde-417e-97c8-6d7627c62c84 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:57.788091 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e39f18-e1c6-4db1-bb2c-5bd2b59953d0 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:57.788647 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e39f18-e1c6-4db1-bb2c-5bd2b59953d0 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:57.789122 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e39f18-e1c6-4db1-bb2c-5bd2b59953d0 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:57.791042 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 226/433] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 05:05:57.796628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d7d81-b08f-4517-9f2b-3e7890301c84 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:57.797207 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d7d81-b08f-4517-9f2b-3e7890301c84 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:57.797701 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d7d81-b08f-4517-9f2b-3e7890301c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:57.817606 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-5a4d7d81-b08f-4517-9f2b-3e7890301c84 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:57.819342 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5a4d7d81-b08f-4517-9f2b-3e7890301c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:57.895137 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa51c75-ddde-417e-97c8-6d7627c62c84 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:57.898739 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 227/434] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 05:05:57.900778 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d7d81-b08f-4517-9f2b-3e7890301c84 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:57.911559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:57.935162 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4994aabb-6a68-41ff-869a-4adfca6bb3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:57.935776 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994aabb-6a68-41ff-869a-4adfca6bb3ae None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:57.936094 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994aabb-6a68-41ff-869a-4adfca6bb3ae None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:57.936638 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994aabb-6a68-41ff-869a-4adfca6bb3ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:57.937232 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4994aabb-6a68-41ff-869a-4adfca6bb3ae None admin] RBAC: Adding query filter params (group.id=89d78f08484744609eb0db0cdb5c9d27, role.id=a74bf01dd66f49f5988a197da269c822, scope.project.id=f9d86697e3eb4fd6b8f82a013cfcb5ba) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:57.937542 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4994aabb-6a68-41ff-869a-4adfca6bb3ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:57.939164 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4994aabb-6a68-41ff-869a-4adfca6bb3ae None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:57.950036 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 208/435] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Aug 31 05:05:57 2022] GET /identity/v3/role_assignments?group.id=89d78f08484744609eb0db0cdb5c9d27&scope.project.id=f9d86697e3eb4fd6b8f82a013cfcb5ba&role.id=a74bf01dd66f49f5988a197da269c822 => generated 611 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:57.998227 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97251d4-d226-4e94-b6fd-cbed5e04db84 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:57.998543 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97251d4-d226-4e94-b6fd-cbed5e04db84 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:57.998906 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97251d4-d226-4e94-b6fd-cbed5e04db84 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:58.001892 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 209/436] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:05:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:05:58.006537 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2979c4-97e6-4160-833a-e68cc85de693 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:58.006891 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2979c4-97e6-4160-833a-e68cc85de693 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:58.007170 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2979c4-97e6-4160-833a-e68cc85de693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:58.031054 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-4b2979c4-97e6-4160-833a-e68cc85de693 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:05:58.032833 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-4b2979c4-97e6-4160-833a-e68cc85de693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:05:58.037370 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:58.061189 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d7d81-b08f-4517-9f2b-3e7890301c84 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:58.108005 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d282f4-827e-46f1-a176-ab970c946bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:58.108798 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d282f4-827e-46f1-a176-ab970c946bc2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:58.109074 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d282f4-827e-46f1-a176-ab970c946bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:58.109340 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d282f4-827e-46f1-a176-ab970c946bc2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:58.123572 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d282f4-827e-46f1-a176-ab970c946bc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:58.129780 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-e1d282f4-827e-46f1-a176-ab970c946bc2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 05:05:58.129780 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:58.129780 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:58.129780 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:58.129780 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:58.129780 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:58.129780 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:58.129780 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:58.129780 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:58.129780 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:58.129780 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:58.129780 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:58.129780 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:58.129780 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:58.129780 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:58.129780 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:58.130684 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:58.130684 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:58.130684 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:58.130684 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:58.130684 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:58.130684 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:58.130684 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:58.130684 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:58.130684 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:58.130684 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:58.130684 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:58.130684 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:58.130684 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:58.130684 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:58.130684 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:58.130684 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:58.131494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:58.131494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:58.131494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:58.131494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:58.131494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:58.131494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:58.131494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:58.131494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:58.131494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:58.131494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:58.131494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 05:05:58.131494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:05:58.132097 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 228/437] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:05:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:05:58.137322 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d282f4-827e-46f1-a176-ab970c946bc2 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:58.141919 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 210/438] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:05:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 05:05:58.186403 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2979c4-97e6-4160-833a-e68cc85de693 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:58.219978 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:58.260895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d282f4-827e-46f1-a176-ab970c946bc2 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:58.306234 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b694ce-d817-4e9c-a28f-62d67af7c752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:58.306850 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b694ce-d817-4e9c-a28f-62d67af7c752 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:58.307125 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b694ce-d817-4e9c-a28f-62d67af7c752 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:58.307806 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b694ce-d817-4e9c-a28f-62d67af7c752 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:58.308493 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b694ce-d817-4e9c-a28f-62d67af7c752 None admin] RBAC: Adding query filter params (name=member) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:58.309051 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b694ce-d817-4e9c-a28f-62d67af7c752 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:58.310513 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b694ce-d817-4e9c-a28f-62d67af7c752 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:58.320562 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 229/439] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 05:05:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:58.320839 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2979c4-97e6-4160-833a-e68cc85de693 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:58.327552 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b694ce-d817-4e9c-a28f-62d67af7c752 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:58.364523 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5c551a-20ef-4963-b672-c6cc81e42845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:58.365761 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5c551a-20ef-4963-b672-c6cc81e42845 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:58.365761 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5c551a-20ef-4963-b672-c6cc81e42845 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:58.365761 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5c551a-20ef-4963-b672-c6cc81e42845 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:58.373026 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:58.378783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5c551a-20ef-4963-b672-c6cc81e42845 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:58.384667 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-5c5c551a-20ef-4963-b672-c6cc81e42845 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:05:58.384667 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:58.384667 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:58.384667 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:58.384667 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:58.384667 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:58.384667 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:58.384667 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:58.384667 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:58.384667 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:58.384667 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:58.384667 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:58.384667 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:58.384667 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:58.384667 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:58.384667 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:58.385594 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:58.385594 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:58.385594 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:58.385594 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:58.385594 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:58.385594 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:58.385594 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:58.385594 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:58.385594 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:58.385594 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:58.385594 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:58.385594 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:58.385594 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:58.385594 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:58.385594 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:58.385594 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:58.386489 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:58.386489 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:58.386489 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:58.386489 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:58.386489 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:58.386489 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:58.386489 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:58.386489 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:58.386489 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:58.386489 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:58.386489 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:05:58.386489 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:05:58.386489 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 211/440] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 05:05:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:05:58.395051 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5c551a-20ef-4963-b672-c6cc81e42845 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:58.467671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b694ce-d817-4e9c-a28f-62d67af7c752 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:58.496182 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:58.505702 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472fec5d-2ef4-4883-9005-9c1d64de91cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:58.506339 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472fec5d-2ef4-4883-9005-9c1d64de91cd None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:58.506619 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472fec5d-2ef4-4883-9005-9c1d64de91cd None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:58.506891 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472fec5d-2ef4-4883-9005-9c1d64de91cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:58.517379 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5c551a-20ef-4963-b672-c6cc81e42845 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:58.551073 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6795fd-5041-4d3b-96ee-04a77011750f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:58.551672 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6795fd-5041-4d3b-96ee-04a77011750f None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:58.552063 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6795fd-5041-4d3b-96ee-04a77011750f None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:58.552186 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6795fd-5041-4d3b-96ee-04a77011750f None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:58.552827 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6795fd-5041-4d3b-96ee-04a77011750f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:58.553126 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6795fd-5041-4d3b-96ee-04a77011750f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:58.554544 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6795fd-5041-4d3b-96ee-04a77011750f None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:58.562653 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 212/441] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:05:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:05:58.563432 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6795fd-5041-4d3b-96ee-04a77011750f None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:58.580026 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472fec5d-2ef4-4883-9005-9c1d64de91cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:58.581483 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472fec5d-2ef4-4883-9005-9c1d64de91cd None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:58.625546 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:58.667900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6795fd-5041-4d3b-96ee-04a77011750f None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:58.681426 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 230/442] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:05:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 05:05:58.686335 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472fec5d-2ef4-4883-9005-9c1d64de91cd None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:58.700645 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2749e30-4cc4-45b2-8964-b5c7e44d85a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:58.701269 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2749e30-4cc4-45b2-8964-b5c7e44d85a6 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:58.701548 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2749e30-4cc4-45b2-8964-b5c7e44d85a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:58.701861 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2749e30-4cc4-45b2-8964-b5c7e44d85a6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:58.711889 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2749e30-4cc4-45b2-8964-b5c7e44d85a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:05:58.712996 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:58.717320 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-f2749e30-4cc4-45b2-8964-b5c7e44d85a6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 05:05:58.717320 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:05:58.717320 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:05:58.717320 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:05:58.717320 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:05:58.717320 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:05:58.717320 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:05:58.717320 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:05:58.717320 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:05:58.717320 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:05:58.717320 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:05:58.717320 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:05:58.717320 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:05:58.717320 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:05:58.717320 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:05:58.717320 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:05:58.718263 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:05:58.718263 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:05:58.718263 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:05:58.718263 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:05:58.718263 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:05:58.718263 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:05:58.718263 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:05:58.718263 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:05:58.718263 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:05:58.718263 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:05:58.718263 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:05:58.718263 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:05:58.718263 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:05:58.718263 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:05:58.718263 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:05:58.718263 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:05:58.719074 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:05:58.719074 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:05:58.719074 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:05:58.719074 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:05:58.719074 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:05:58.719074 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:05:58.719074 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:05:58.719074 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:05:58.719074 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:05:58.719074 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:05:58.719074 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 05:05:58.719074 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:05:58.719074 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 213/443] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 05:05:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:05:58.719753 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2749e30-4cc4-45b2-8964-b5c7e44d85a6 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:58.786080 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472fec5d-2ef4-4883-9005-9c1d64de91cd None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:58.816523 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2749e30-4cc4-45b2-8964-b5c7e44d85a6 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:58.819104 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d397f0-340f-4dc8-a50b-94d4b62ce4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:58.819703 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d397f0-340f-4dc8-a50b-94d4b62ce4e8 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:58.819978 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d397f0-340f-4dc8-a50b-94d4b62ce4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:58.820245 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d397f0-340f-4dc8-a50b-94d4b62ce4e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:58.836197 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:58.850525 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6755185-880e-4a64-8d7e-801dd57d910b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:58.851147 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6755185-880e-4a64-8d7e-801dd57d910b None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:58.851426 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6755185-880e-4a64-8d7e-801dd57d910b None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:58.851697 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6755185-880e-4a64-8d7e-801dd57d910b None admin] PATH_INFO: `/v3/projects/b08ea5fe820543b6936ffde78e21ea4f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:58.852846 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6755185-880e-4a64-8d7e-801dd57d910b None admin] RBAC: Authorizing `identity:get_project(project_id=b08ea5fe820543b6936ffde78e21ea4f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:58.854332 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6755185-880e-4a64-8d7e-801dd57d910b None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:58.855922 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 214/444] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:05:58 2022] GET /identity/v3/projects/b08ea5fe820543b6936ffde78e21ea4f => generated 302 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:58.856673 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6755185-880e-4a64-8d7e-801dd57d910b None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:58.891299 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d397f0-340f-4dc8-a50b-94d4b62ce4e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:58.892960 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d397f0-340f-4dc8-a50b-94d4b62ce4e8 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:58.921919 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:58.962316 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6755185-880e-4a64-8d7e-801dd57d910b None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:58.993133 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 231/445] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:05:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 05:05:58.994309 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d397f0-340f-4dc8-a50b-94d4b62ce4e8 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:59.000209 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8a67a9-bdf4-4e1a-88a1-345b54c7533d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:59.000819 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a67a9-bdf4-4e1a-88a1-345b54c7533d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:59.001095 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a67a9-bdf4-4e1a-88a1-345b54c7533d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:59.001360 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a67a9-bdf4-4e1a-88a1-345b54c7533d None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:59.002033 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8a67a9-bdf4-4e1a-88a1-345b54c7533d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:59.002349 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8a67a9-bdf4-4e1a-88a1-345b54c7533d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:59.003718 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8a67a9-bdf4-4e1a-88a1-345b54c7533d None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:59.011813 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 215/446] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:05:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:59.012764 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8a67a9-bdf4-4e1a-88a1-345b54c7533d None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:59.060476 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:59.091427 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d397f0-340f-4dc8-a50b-94d4b62ce4e8 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:59.110230 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8a67a9-bdf4-4e1a-88a1-345b54c7533d None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:59.117411 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:59.126400 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7811576-4b99-451a-aa08-40091630f71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:59.127032 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7811576-4b99-451a-aa08-40091630f71f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:59.127306 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7811576-4b99-451a-aa08-40091630f71f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:59.127572 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7811576-4b99-451a-aa08-40091630f71f None admin] PATH_INFO: `/v3/groups/89d78f08484744609eb0db0cdb5c9d27` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:59.128679 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7811576-4b99-451a-aa08-40091630f71f None admin] RBAC: Authorizing `identity:get_group(group_id=89d78f08484744609eb0db0cdb5c9d27)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:59.130158 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7811576-4b99-451a-aa08-40091630f71f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:59.131726 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 232/447] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Aug 31 05:05:58 2022] GET /identity/v3/groups/89d78f08484744609eb0db0cdb5c9d27 => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:59.132511 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7811576-4b99-451a-aa08-40091630f71f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:59.144453 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29699831-8698-4864-86ca-cd8b78748eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:59.145152 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29699831-8698-4864-86ca-cd8b78748eb5 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:59.145424 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29699831-8698-4864-86ca-cd8b78748eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:59.145687 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29699831-8698-4864-86ca-cd8b78748eb5 None admin] PATH_INFO: `/v3/projects/3e3bdb87df7f4087806a21456cb45a01` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:59.146821 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29699831-8698-4864-86ca-cd8b78748eb5 None admin] RBAC: Authorizing `identity:get_project(project_id=3e3bdb87df7f4087806a21456cb45a01)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:59.148246 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29699831-8698-4864-86ca-cd8b78748eb5 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:59.149845 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 216/448] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:05:59 2022] GET /identity/v3/projects/3e3bdb87df7f4087806a21456cb45a01 => generated 344 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:59.150630 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29699831-8698-4864-86ca-cd8b78748eb5 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:59.227959 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7811576-4b99-451a-aa08-40091630f71f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:59.248352 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29699831-8698-4864-86ca-cd8b78748eb5 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:59.261071 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76549b6-638b-459e-986a-d8ec0d181fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:59.261661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76549b6-638b-459e-986a-d8ec0d181fa9 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:59.261935 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76549b6-638b-459e-986a-d8ec0d181fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:59.262230 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76549b6-638b-459e-986a-d8ec0d181fa9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:59.267774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:59.281485 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b5430e-8f7a-4194-8976-312bc3206c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:59.282107 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5430e-8f7a-4194-8976-312bc3206c0e None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:59.282383 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5430e-8f7a-4194-8976-312bc3206c0e None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:59.282653 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5430e-8f7a-4194-8976-312bc3206c0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:59.283381 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b5430e-8f7a-4194-8976-312bc3206c0e None admin] RBAC: Adding query filter params (group.id=89d78f08484744609eb0db0cdb5c9d27, role.id=e3235eda7c654e2abf96ed6a494f5152, scope.project.id=b08ea5fe820543b6936ffde78e21ea4f) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:59.283685 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b5430e-8f7a-4194-8976-312bc3206c0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:59.285124 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b5430e-8f7a-4194-8976-312bc3206c0e None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:59.290711 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 217/449] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Aug 31 05:05:59 2022] GET /identity/v3/role_assignments?group.id=89d78f08484744609eb0db0cdb5c9d27&scope.project.id=b08ea5fe820543b6936ffde78e21ea4f&role.id=e3235eda7c654e2abf96ed6a494f5152 => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:59.291469 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b5430e-8f7a-4194-8976-312bc3206c0e None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:59.328591 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:59.336380 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76549b6-638b-459e-986a-d8ec0d181fa9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:59.337861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76549b6-638b-459e-986a-d8ec0d181fa9 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:59.413146 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b5430e-8f7a-4194-8976-312bc3206c0e None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:59.453004 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 233/450] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:05:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 05:05:59.456214 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a74d16-8a77-42ea-8cb8-0b37ba5c96b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:59.456864 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a74d16-8a77-42ea-8cb8-0b37ba5c96b5 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:59.457156 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a74d16-8a77-42ea-8cb8-0b37ba5c96b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:59.457397 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a74d16-8a77-42ea-8cb8-0b37ba5c96b5 None admin] PATH_INFO: `/v3/groups/024e4c11d0144a6ba8107c6d169605ce` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:59.458545 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a74d16-8a77-42ea-8cb8-0b37ba5c96b5 None admin] RBAC: Authorizing `identity:get_group(group_id=024e4c11d0144a6ba8107c6d169605ce)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:59.458780 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76549b6-638b-459e-986a-d8ec0d181fa9 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:59.460007 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a74d16-8a77-42ea-8cb8-0b37ba5c96b5 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:59.461707 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 218/451] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Aug 31 05:05:59 2022] GET /identity/v3/groups/024e4c11d0144a6ba8107c6d169605ce => generated 229 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:05:59.468548 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a74d16-8a77-42ea-8cb8-0b37ba5c96b5 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:59.523148 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:59.563279 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:59.571422 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76549b6-638b-459e-986a-d8ec0d181fa9 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:59.578456 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a74d16-8a77-42ea-8cb8-0b37ba5c96b5 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:59.612737 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f6a701-4d49-44a3-8b96-adba526dad10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:59.613377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f6a701-4d49-44a3-8b96-adba526dad10 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:59.613731 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f6a701-4d49-44a3-8b96-adba526dad10 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:59.614021 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f6a701-4d49-44a3-8b96-adba526dad10 None admin] PATH_INFO: `/v3/projects/b08ea5fe820543b6936ffde78e21ea4f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:59.615979 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f6a701-4d49-44a3-8b96-adba526dad10 None admin] RBAC: Authorizing `identity:get_project(project_id=b08ea5fe820543b6936ffde78e21ea4f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:59.617543 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f6a701-4d49-44a3-8b96-adba526dad10 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:59.618933 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0f37c3-6a28-4307-9a6c-7af2be1538d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:59.619057 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 234/452] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:05:59 2022] GET /identity/v3/projects/b08ea5fe820543b6936ffde78e21ea4f => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:05:59.619265 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f37c3-6a28-4307-9a6c-7af2be1538d9 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:59.619537 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f37c3-6a28-4307-9a6c-7af2be1538d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:59.619838 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f37c3-6a28-4307-9a6c-7af2be1538d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:59.620517 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0f37c3-6a28-4307-9a6c-7af2be1538d9 None admin] RBAC: Adding query filter params (group.id=024e4c11d0144a6ba8107c6d169605ce, role.id=c3d5589dd2864be9a0f26f5802d874cd, scope.project.id=3e3bdb87df7f4087806a21456cb45a01) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:59.620816 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0f37c3-6a28-4307-9a6c-7af2be1538d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:59.622256 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0f37c3-6a28-4307-9a6c-7af2be1538d9 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:59.624383 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f6a701-4d49-44a3-8b96-adba526dad10 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:59.630032 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 219/453] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Aug 31 05:05:59 2022] GET /identity/v3/role_assignments?group.id=024e4c11d0144a6ba8107c6d169605ce&scope.project.id=3e3bdb87df7f4087806a21456cb45a01&role.id=c3d5589dd2864be9a0f26f5802d874cd => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:05:59.737849 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f6a701-4d49-44a3-8b96-adba526dad10 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:59.774983 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95101004-c961-498b-a40b-eb9eb19b13ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:59.775605 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95101004-c961-498b-a40b-eb9eb19b13ec None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:59.775877 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95101004-c961-498b-a40b-eb9eb19b13ec None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:59.776137 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95101004-c961-498b-a40b-eb9eb19b13ec None admin] PATH_INFO: `/v3/groups/89d78f08484744609eb0db0cdb5c9d27` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:59.777252 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95101004-c961-498b-a40b-eb9eb19b13ec None admin] RBAC: Authorizing `identity:get_group(group_id=89d78f08484744609eb0db0cdb5c9d27)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:59.778732 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95101004-c961-498b-a40b-eb9eb19b13ec None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:59.780391 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 235/454] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Aug 31 05:05:59 2022] GET /identity/v3/groups/89d78f08484744609eb0db0cdb5c9d27 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:05:59.786394 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0f37c3-6a28-4307-9a6c-7af2be1538d9 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:05:59.888569 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0f37c3-6a28-4307-9a6c-7af2be1538d9 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:05:59.905506 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:05:59.927250 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd38c7-d80d-4225-97e1-fcf769695df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:05:59.927852 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd38c7-d80d-4225-97e1-fcf769695df8 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:05:59.928129 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd38c7-d80d-4225-97e1-fcf769695df8 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:05:59.928391 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd38c7-d80d-4225-97e1-fcf769695df8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:05:59.929119 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cd38c7-d80d-4225-97e1-fcf769695df8 None admin] RBAC: Adding query filter params (group.id=89d78f08484744609eb0db0cdb5c9d27, role.id=a74bf01dd66f49f5988a197da269c822, scope.project.id=b08ea5fe820543b6936ffde78e21ea4f) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:05:59.929424 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cd38c7-d80d-4225-97e1-fcf769695df8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:05:59.930916 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cd38c7-d80d-4225-97e1-fcf769695df8 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:05:59.936630 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 220/455] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Aug 31 05:05:59 2022] GET /identity/v3/role_assignments?group.id=89d78f08484744609eb0db0cdb5c9d27&scope.project.id=b08ea5fe820543b6936ffde78e21ea4f&role.id=a74bf01dd66f49f5988a197da269c822 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:02.297763 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998d294-4199-4ebd-977a-a65e4e7d926d None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:02.298846 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998d294-4199-4ebd-977a-a65e4e7d926d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:02.299326 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998d294-4199-4ebd-977a-a65e4e7d926d None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:02.301121 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 236/456] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:02.306918 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526e80a-ed53-4a53-a1e5-2cd2ece7cccc None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:02.307750 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526e80a-ed53-4a53-a1e5-2cd2ece7cccc None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:02.308239 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526e80a-ed53-4a53-a1e5-2cd2ece7cccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:02.334981 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-f526e80a-ed53-4a53-a1e5-2cd2ece7cccc None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:02.336513 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-f526e80a-ed53-4a53-a1e5-2cd2ece7cccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:02.431671 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 221/457] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 05:06:02.458669 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f2227-cfe7-4d1f-8187-d2f3445d4240 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:02.459017 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f2227-cfe7-4d1f-8187-d2f3445d4240 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:02.459206 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f2227-cfe7-4d1f-8187-d2f3445d4240 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:02.460787 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 237/458] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:02.465978 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5beca-5257-41d3-a980-5ef7795aeceb None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:02.466285 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5beca-5257-41d3-a980-5ef7795aeceb None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:02.466554 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5beca-5257-41d3-a980-5ef7795aeceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:02.492655 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-b3b5beca-5257-41d3-a980-5ef7795aeceb None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:02.494190 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b3b5beca-5257-41d3-a980-5ef7795aeceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:02.576347 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 222/459] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:06:02.736620 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f2227-cfe7-4d1f-8187-d2f3445d4240 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:02.774151 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:02.871324 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f2227-cfe7-4d1f-8187-d2f3445d4240 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:02.917190 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a61b0-544c-4cfc-aa72-96d884b740ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:02.917829 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a61b0-544c-4cfc-aa72-96d884b740ee None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:02.918158 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a61b0-544c-4cfc-aa72-96d884b740ee None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:02.918434 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a61b0-544c-4cfc-aa72-96d884b740ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:02.929616 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a61b0-544c-4cfc-aa72-96d884b740ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:02.931107 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a61b0-544c-4cfc-aa72-96d884b740ee None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:02.947324 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b5beca-5257-41d3-a980-5ef7795aeceb None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:02.955755 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-4f6a61b0-544c-4cfc-aa72-96d884b740ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:02.955755 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:02.955755 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:02.955755 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:02.955755 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:02.955755 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:02.955755 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:02.955755 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:02.955755 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:02.955755 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:02.955755 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:02.955755 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:02.955755 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:02.955755 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:02.955755 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:02.955755 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:02.956671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:02.956671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:02.956671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:02.956671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:02.956671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:02.956671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:02.956671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:02.956671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:02.956671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:02.956671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:02.956671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:02.956671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:02.956671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:02.956671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:02.956671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:02.956671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:02.956671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:02.957925 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:02.957925 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:02.957925 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:02.957925 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 238/460] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:02.964126 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a61b0-544c-4cfc-aa72-96d884b740ee None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:02.987084 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:03.048655 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b5beca-5257-41d3-a980-5ef7795aeceb None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:03.057743 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:03.082341 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a61b0-544c-4cfc-aa72-96d884b740ee None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:03.086589 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dccc5a-807c-4cfb-86b9-8ece035442de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:03.087330 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dccc5a-807c-4cfb-86b9-8ece035442de None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:03.087699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dccc5a-807c-4cfb-86b9-8ece035442de None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:03.088065 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dccc5a-807c-4cfb-86b9-8ece035442de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:03.099003 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dccc5a-807c-4cfb-86b9-8ece035442de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:03.100602 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dccc5a-807c-4cfb-86b9-8ece035442de None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:03.115855 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-34dccc5a-807c-4cfb-86b9-8ece035442de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:03.115855 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:03.115855 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:03.115855 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:03.115855 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:03.115855 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:03.115855 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:03.115855 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:03.115855 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:03.115855 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:03.115855 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:03.115855 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:03.115855 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:03.115855 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:03.115855 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:03.115855 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:03.117117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:03.117117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:03.117117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:03.117117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:03.117117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:03.117117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:03.117117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:03.117117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:03.117117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:03.117117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:03.117117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:03.117117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:03.117117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:03.117117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:03.117117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:03.117117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:03.117117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:03.118915 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:03.118915 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:03.118915 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:03.124623 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 223/461] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:03.125977 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e221bb9-a655-4de6-a039-9306ad7f29c8 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:03.126377 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e221bb9-a655-4de6-a039-9306ad7f29c8 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:03.126738 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e221bb9-a655-4de6-a039-9306ad7f29c8 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:03.128348 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 224/462] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:03.129153 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e221bb9-a655-4de6-a039-9306ad7f29c8 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:03.129386 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242dd99-855e-4d8b-aba9-f6da5d4cebb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:03.130185 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242dd99-855e-4d8b-aba9-f6da5d4cebb7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:03.130605 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242dd99-855e-4d8b-aba9-f6da5d4cebb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:03.131012 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242dd99-855e-4d8b-aba9-f6da5d4cebb7 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:03.131820 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6242dd99-855e-4d8b-aba9-f6da5d4cebb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:03.132284 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6242dd99-855e-4d8b-aba9-f6da5d4cebb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:03.134915 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6242dd99-855e-4d8b-aba9-f6da5d4cebb7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:03.153750 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 239/463] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:03.155571 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42821545-5c94-4868-a704-5801f337ce32 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:03.156120 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42821545-5c94-4868-a704-5801f337ce32 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:03.157292 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42821545-5c94-4868-a704-5801f337ce32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:03.176299 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-42821545-5c94-4868-a704-5801f337ce32 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:03.177493 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-42821545-5c94-4868-a704-5801f337ce32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:03.229298 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:03.241177 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e221bb9-a655-4de6-a039-9306ad7f29c8 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:03.249437 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 240/464] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:06:03.249437 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d237cc-dcc8-4ea8-9bf0-66e458f80742 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:03.249437 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d237cc-dcc8-4ea8-9bf0-66e458f80742 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:03.249437 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d237cc-dcc8-4ea8-9bf0-66e458f80742 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:03.249437 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 241/465] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:03.249437 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d237cc-dcc8-4ea8-9bf0-66e458f80742 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:03.285674 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dec993-1627-4d49-83c0-2a1a78b2e36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:03.286398 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dec993-1627-4d49-83c0-2a1a78b2e36f None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:03.286825 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dec993-1627-4d49-83c0-2a1a78b2e36f None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:03.287158 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dec993-1627-4d49-83c0-2a1a78b2e36f None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:03.288004 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dec993-1627-4d49-83c0-2a1a78b2e36f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:03.288384 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dec993-1627-4d49-83c0-2a1a78b2e36f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:03.289758 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dec993-1627-4d49-83c0-2a1a78b2e36f None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:03.307927 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 225/466] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:03.314050 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af550c57-fd9b-43b7-8599-c2256030dc3b None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:03.314324 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af550c57-fd9b-43b7-8599-c2256030dc3b None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:03.314601 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af550c57-fd9b-43b7-8599-c2256030dc3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:03.336455 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-af550c57-fd9b-43b7-8599-c2256030dc3b None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:03.337742 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-af550c57-fd9b-43b7-8599-c2256030dc3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:03.340040 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:03.372705 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d237cc-dcc8-4ea8-9bf0-66e458f80742 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:03.417619 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dd71f0-dafb-45da-93ef-f27f34639c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:03.418167 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd71f0-dafb-45da-93ef-f27f34639c31 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:03.418443 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd71f0-dafb-45da-93ef-f27f34639c31 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:03.418711 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd71f0-dafb-45da-93ef-f27f34639c31 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:03.419375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dd71f0-dafb-45da-93ef-f27f34639c31 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:03.420873 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dd71f0-dafb-45da-93ef-f27f34639c31 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:03.426750 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 226/467] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:06:03.427830 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af550c57-fd9b-43b7-8599-c2256030dc3b None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:03.460919 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 242/468] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Aug 31 05:06:03 2022] POST /identity/v3/users => generated 246 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:06:03.519281 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:03.540106 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af550c57-fd9b-43b7-8599-c2256030dc3b None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:03.576871 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea93044-62da-47e0-90fb-db4097330331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:03.577866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea93044-62da-47e0-90fb-db4097330331 None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:03.578361 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea93044-62da-47e0-90fb-db4097330331 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:03.578812 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea93044-62da-47e0-90fb-db4097330331 None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:03.579671 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea93044-62da-47e0-90fb-db4097330331 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:03.581309 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea93044-62da-47e0-90fb-db4097330331 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:03.598180 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dd71f0-dafb-45da-93ef-f27f34639c31 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:03.610430 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 227/469] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Aug 31 05:06:03 2022] POST /identity/v3/users => generated 248 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:06:03.763674 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dd71f0-dafb-45da-93ef-f27f34639c31 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:03.765563 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea93044-62da-47e0-90fb-db4097330331 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:03.801767 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:03.812689 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f0786-7d99-465d-a227-53ca338aa2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:03.813340 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f0786-7d99-465d-a227-53ca338aa2e0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:03.813613 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f0786-7d99-465d-a227-53ca338aa2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:03.813878 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f0786-7d99-465d-a227-53ca338aa2e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:03.830665 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2f0786-7d99-465d-a227-53ca338aa2e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:03.832199 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2f0786-7d99-465d-a227-53ca338aa2e0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:03.851138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-de2f0786-7d99-465d-a227-53ca338aa2e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:03.851138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:03.851138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:03.851138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:03.851138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:03.851138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:03.851138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:03.851138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:03.851138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:03.851138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:03.851138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:03.851138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:03.851138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:03.851138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:03.851138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:03.851138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:03.852075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:03.852075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:03.852075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:03.852075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:03.852075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:03.852075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:03.852075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:03.852075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:03.852075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:03.852075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:03.852075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:03.852075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:03.852075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:03.852075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:03.852075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:03.852075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:03.852075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:03.852923 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:03.852923 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:03.852923 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:03.852923 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 243/470] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:06:03.858459 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f0786-7d99-465d-a227-53ca338aa2e0 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:03.911501 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea93044-62da-47e0-90fb-db4097330331 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:03.962335 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f9b7c-1e18-48e9-87ff-cd9cb8032a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:03.963077 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f9b7c-1e18-48e9-87ff-cd9cb8032a7a None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:03.963421 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f9b7c-1e18-48e9-87ff-cd9cb8032a7a None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:03.963762 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f9b7c-1e18-48e9-87ff-cd9cb8032a7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:03.967177 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:03.978896 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069f9b7c-1e18-48e9-87ff-cd9cb8032a7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:03.980529 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069f9b7c-1e18-48e9-87ff-cd9cb8032a7a None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:03.981513 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f0786-7d99-465d-a227-53ca338aa2e0 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:03.996077 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-069f9b7c-1e18-48e9-87ff-cd9cb8032a7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:03.996077 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:03.996077 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:03.996077 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:03.996077 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:03.996077 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:03.996077 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:03.996077 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:03.996077 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:03.996077 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:03.996077 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:03.996077 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:03.996077 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:03.996077 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:03.996077 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:03.996077 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:03.997007 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:03.997007 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:03.997007 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:03.997007 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:03.997007 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:03.997007 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:03.997007 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:03.997007 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:03.997007 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:03.997007 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:03.997007 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:03.997007 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:03.997007 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:03.997007 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:03.997007 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:03.997007 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:03.997007 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:03.997871 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:03.997871 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:03.997871 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:04.006513 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 228/471] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:04.007509 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f9b7c-1e18-48e9-87ff-cd9cb8032a7a None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:04.020780 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bda6e5-9a19-4fb6-93a2-d3317d66b8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:04.021320 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bda6e5-9a19-4fb6-93a2-d3317d66b8d3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:04.021598 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bda6e5-9a19-4fb6-93a2-d3317d66b8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:04.021870 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bda6e5-9a19-4fb6-93a2-d3317d66b8d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:04.022557 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bda6e5-9a19-4fb6-93a2-d3317d66b8d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:04.022858 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bda6e5-9a19-4fb6-93a2-d3317d66b8d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:04.024243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bda6e5-9a19-4fb6-93a2-d3317d66b8d3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:04.039542 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 244/472] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:04.044796 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bda6e5-9a19-4fb6-93a2-d3317d66b8d3 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:04.123214 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:04.159758 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f9b7c-1e18-48e9-87ff-cd9cb8032a7a None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:04.183237 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bda6e5-9a19-4fb6-93a2-d3317d66b8d3 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:04.204944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d038f01-5195-4714-89ff-5d18202a7cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:04.206426 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d038f01-5195-4714-89ff-5d18202a7cda None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:04.206934 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d038f01-5195-4714-89ff-5d18202a7cda None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:04.207643 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d038f01-5195-4714-89ff-5d18202a7cda None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:04.208736 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d038f01-5195-4714-89ff-5d18202a7cda None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:04.209628 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d038f01-5195-4714-89ff-5d18202a7cda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:04.211763 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d038f01-5195-4714-89ff-5d18202a7cda None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:04.220186 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ce1e5-95ed-48d0-8467-48a45ffc5d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:04.220792 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ce1e5-95ed-48d0-8467-48a45ffc5d54 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:04.221062 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ce1e5-95ed-48d0-8467-48a45ffc5d54 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:04.221326 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ce1e5-95ed-48d0-8467-48a45ffc5d54 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:04.222118 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229ce1e5-95ed-48d0-8467-48a45ffc5d54 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:04.223409 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229ce1e5-95ed-48d0-8467-48a45ffc5d54 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:04.227379 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 229/473] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:04.228827 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6a734-6d4b-4829-953a-98c12f737c99 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:04.229316 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6a734-6d4b-4829-953a-98c12f737c99 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:04.229770 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6a734-6d4b-4829-953a-98c12f737c99 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:04.231417 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 230/474] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:04.240511 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a6a734-6d4b-4829-953a-98c12f737c99 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:04.261831 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 245/475] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Aug 31 05:06:04 2022] POST /identity/v3/users => generated 248 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:06:04.263352 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47431010-b826-4616-aaef-fd2d988a0da3 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:04.264233 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47431010-b826-4616-aaef-fd2d988a0da3 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:04.264233 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47431010-b826-4616-aaef-fd2d988a0da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:04.296845 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-47431010-b826-4616-aaef-fd2d988a0da3 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:04.302510 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-47431010-b826-4616-aaef-fd2d988a0da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:04.304420 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:04.356331 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a6a734-6d4b-4829-953a-98c12f737c99 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:04.400155 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd186bb4-0476-4d78-b6b6-5e8e088449e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:04.401281 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd186bb4-0476-4d78-b6b6-5e8e088449e0 None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:04.401801 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd186bb4-0476-4d78-b6b6-5e8e088449e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:04.402358 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd186bb4-0476-4d78-b6b6-5e8e088449e0 None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:04.403268 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd186bb4-0476-4d78-b6b6-5e8e088449e0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:04.406740 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd186bb4-0476-4d78-b6b6-5e8e088449e0 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:04.434999 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 246/476] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:06:04.449798 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 231/477] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Aug 31 05:06:04 2022] POST /identity/v3/users => generated 249 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:06:04.769052 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47431010-b826-4616-aaef-fd2d988a0da3 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:04.800038 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:04.889165 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47431010-b826-4616-aaef-fd2d988a0da3 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:04.939452 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65fecf9-30c4-409d-818b-11120e52414d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:04.940005 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65fecf9-30c4-409d-818b-11120e52414d None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:04.940286 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65fecf9-30c4-409d-818b-11120e52414d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:04.940555 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65fecf9-30c4-409d-818b-11120e52414d None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:04.941248 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65fecf9-30c4-409d-818b-11120e52414d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:04.942665 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65fecf9-30c4-409d-818b-11120e52414d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:04.955183 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.sql.core [None req-d65fecf9-30c4-409d-818b-11120e52414d None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 05:06:04.955183 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 31 05:06:04.955183 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [parameters: {'id': 'bc86a630a5d44839b4014160ac6f805c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 31 05:06:04.955183 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81602) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 05:06:04.976748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-d65fecf9-30c4-409d-818b-11120e52414d None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 31 05:06:04.976748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:04.976748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 05:06:04.976748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 05:06:04.976748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 05:06:04.976748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 05:06:04.976748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 05:06:04.976748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 05:06:04.976748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 05:06:04.976748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application conn.query(q) Aug 31 05:06:04.976748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 05:06:04.976748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 05:06:04.976748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 05:06:04.976748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application result.read() Aug 31 05:06:04.976748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 05:06:04.977724 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 05:06:04.977724 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 05:06:04.977724 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 05:06:04.977724 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 05:06:04.977724 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 05:06:04.977724 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 05:06:04.977724 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 05:06:04.977724 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 05:06:04.977724 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:04.977724 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 05:06:04.977724 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:04.977724 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:04.977724 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 05:06:04.977724 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 05:06:04.977724 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 31 05:06:04.977724 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return ref.to_dict() Aug 31 05:06:04.977724 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 05:06:04.977724 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application next(self.gen) Aug 31 05:06:04.979123 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 05:06:04.979123 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application yield resource Aug 31 05:06:04.979123 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 05:06:04.979123 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application next(self.gen) Aug 31 05:06:04.979123 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 05:06:04.979123 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 05:06:04.979123 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 05:06:04.979123 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 05:06:04.979123 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 05:06:04.979123 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise self.value Aug 31 05:06:04.979123 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 05:06:04.979123 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 05:06:04.979123 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 05:06:04.979123 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application session.commit() Aug 31 05:06:04.979123 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 05:06:04.979123 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 05:06:04.979123 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 05:06:04.980181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 05:06:04.980181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 05:06:04.980181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self.session.flush() Aug 31 05:06:04.980181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 05:06:04.980181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 05:06:04.980181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 05:06:04.980181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 05:06:04.980181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 05:06:04.980181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application compat.raise_( Aug 31 05:06:04.980181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 05:06:04.980181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception Aug 31 05:06:04.980181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 05:06:04.980181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 05:06:04.980181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 05:06:04.980181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 05:06:04.980181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 05:06:04.980181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 05:06:04.981067 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 05:06:04.981067 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 05:06:04.981067 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 31 05:06:04.981067 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 31 05:06:04.981067 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 05:06:04.981067 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 05:06:04.981067 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 05:06:04.981067 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 05:06:04.981067 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 05:06:04.981067 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 05:06:04.981067 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 05:06:04.981067 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 05:06:04.981067 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 05:06:04.981067 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 05:06:04.981067 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 05:06:04.981067 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception Aug 31 05:06:04.981876 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 05:06:04.981876 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 05:06:04.981876 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 05:06:04.981876 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 05:06:04.981876 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 05:06:04.981876 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 05:06:04.981876 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 05:06:04.981876 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application conn.query(q) Aug 31 05:06:04.981876 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 05:06:04.981876 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 05:06:04.981876 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 05:06:04.981876 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application result.read() Aug 31 05:06:04.981876 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 05:06:04.981876 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 05:06:04.981876 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 05:06:04.981876 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 05:06:04.983823 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 05:06:04.983823 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 05:06:04.983823 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 05:06:04.983823 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 05:06:04.983823 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 05:06:04.983823 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 31 05:06:04.983823 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [parameters: {'id': 'bc86a630a5d44839b4014160ac6f805c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 31 05:06:04.983823 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 05:06:04.983823 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:04.983823 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 05:06:04.983823 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:04.983823 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:04.983823 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:04.983823 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:04.983823 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:04.983823 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:04.985728 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:04.985728 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:04.985728 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:04.985728 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:04.985728 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:04.985728 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:04.985728 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 31 05:06:04.985728 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 31 05:06:04.985728 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:04.985728 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:04.985728 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 31 05:06:04.985728 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 31 05:06:04.985728 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 05:06:04.985728 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 05:06:04.985728 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 31 05:06:04.985728 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:04.987767 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 247/478] 10.222.0.36 () {60 vars in 1131 bytes} [Wed Aug 31 05:06:04 2022] POST /identity/v3/roles => generated 144 bytes in 210 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:04.987902 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd186bb4-0476-4d78-b6b6-5e8e088449e0 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:05.120817 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd186bb4-0476-4d78-b6b6-5e8e088449e0 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:05.123981 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:05.175995 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070832e8-a28b-471c-bb5e-2a0c434051c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:05.176631 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070832e8-a28b-471c-bb5e-2a0c434051c5 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:05.176904 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070832e8-a28b-471c-bb5e-2a0c434051c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:05.177171 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070832e8-a28b-471c-bb5e-2a0c434051c5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:05.191799 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070832e8-a28b-471c-bb5e-2a0c434051c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:06:05.196535 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-070832e8-a28b-471c-bb5e-2a0c434051c5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 05:06:05.196535 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:05.196535 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:05.196535 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:05.196535 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:05.196535 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:05.196535 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:05.196535 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:05.196535 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:05.196535 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:05.196535 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:05.196535 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:05.196535 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:06:05.196535 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:06:05.196535 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:06:05.196535 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:06:05.197404 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:06:05.197404 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:06:05.197404 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:06:05.197404 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:06:05.197404 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:05.197404 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:05.197404 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:05.197404 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:05.197404 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:05.197404 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:05.197404 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:05.197404 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:05.197404 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:05.197404 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:05.197404 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:05.197404 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:05.198851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:05.198851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:05.198851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:05.198851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:05.198851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:06:05.198851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:06:05.198851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:06:05.198851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:06:05.198851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:06:05.198851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:06:05.198851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 05:06:05.198851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:05.198851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 232/479] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 05:06:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:05.203508 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65fecf9-30c4-409d-818b-11120e52414d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:05.313473 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:05.327724 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65fecf9-30c4-409d-818b-11120e52414d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:05.378706 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c755a8-1364-47f3-8469-949cabca47fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:05.379700 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c755a8-1364-47f3-8469-949cabca47fe None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:05.380165 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c755a8-1364-47f3-8469-949cabca47fe None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:05.380639 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c755a8-1364-47f3-8469-949cabca47fe None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:05.381488 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c755a8-1364-47f3-8469-949cabca47fe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:05.381983 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c755a8-1364-47f3-8469-949cabca47fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:05.383563 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c755a8-1364-47f3-8469-949cabca47fe None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:05.395931 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 248/480] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:06:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:05.913178 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4f222-a8cc-4b58-82ea-6968b547ecc7 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:05.913697 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4f222-a8cc-4b58-82ea-6968b547ecc7 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:05.913876 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4f222-a8cc-4b58-82ea-6968b547ecc7 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:05.915377 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 233/481] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:05.921163 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc902e-c66f-439a-807f-ba6f94b5107c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:05.922229 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc902e-c66f-439a-807f-ba6f94b5107c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:05.922719 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc902e-c66f-439a-807f-ba6f94b5107c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:05.942815 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-82fc902e-c66f-439a-807f-ba6f94b5107c None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:05.944590 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-82fc902e-c66f-439a-807f-ba6f94b5107c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:06.000740 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac121f4b-b6ed-4c33-b57b-9015cf6ed4e3 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:06.001044 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac121f4b-b6ed-4c33-b57b-9015cf6ed4e3 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:06.001275 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac121f4b-b6ed-4c33-b57b-9015cf6ed4e3 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:06.002850 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 234/482] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:06.007950 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fcf6d6-0613-410d-8add-fba3d189e919 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:06.008223 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fcf6d6-0613-410d-8add-fba3d189e919 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:06.008492 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fcf6d6-0613-410d-8add-fba3d189e919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:06.032367 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-a5fcf6d6-0613-410d-8add-fba3d189e919 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:06.033552 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a5fcf6d6-0613-410d-8add-fba3d189e919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:06.039542 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 249/483] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:06:06.131586 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 235/484] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 05:06:06.355448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc902e-c66f-439a-807f-ba6f94b5107c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:06.443898 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:06.461753 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc902e-c66f-439a-807f-ba6f94b5107c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:06.471326 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fcf6d6-0613-410d-8add-fba3d189e919 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:06.499443 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b998a7-7605-4fc8-99c9-6df8e5943c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:06.500237 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b998a7-7605-4fc8-99c9-6df8e5943c54 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:06.500625 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b998a7-7605-4fc8-99c9-6df8e5943c54 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:06.500819 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b998a7-7605-4fc8-99c9-6df8e5943c54 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:06.512136 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b998a7-7605-4fc8-99c9-6df8e5943c54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:06:06.517351 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-f4b998a7-7605-4fc8-99c9-6df8e5943c54 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 05:06:06.517351 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:06.517351 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:06.517351 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:06.517351 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:06.517351 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:06.517351 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:06.517351 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:06.517351 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:06.517351 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:06.517351 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:06.517351 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:06.517351 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:06:06.517351 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:06:06.517351 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:06:06.517351 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:06:06.518412 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:06:06.518412 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:06:06.518412 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:06:06.518412 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:06:06.518412 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:06.518412 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:06.518412 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:06.518412 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:06.518412 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:06.518412 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:06.518412 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:06.518412 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:06.518412 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:06.518412 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:06.518412 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:06.518412 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:06.519198 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:06.519198 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:06.519198 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:06.519198 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:06.519198 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:06:06.519198 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:06:06.519198 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:06:06.519198 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:06:06.519198 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:06:06.519198 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:06:06.519198 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 05:06:06.519198 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:06.519797 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 250/485] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 05:06:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:06.526809 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b998a7-7605-4fc8-99c9-6df8e5943c54 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:06.641269 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:06.652818 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fcf6d6-0613-410d-8add-fba3d189e919 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:06.729168 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b998a7-7605-4fc8-99c9-6df8e5943c54 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:06.731307 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3ab19-64c4-4899-8891-ee6018c512ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:06.731947 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3ab19-64c4-4899-8891-ee6018c512ed None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:06.732218 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3ab19-64c4-4899-8891-ee6018c512ed None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:06.732478 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3ab19-64c4-4899-8891-ee6018c512ed None admin] PATH_INFO: `/v3/roles/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:06.753815 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd3ab19-64c4-4899-8891-ee6018c512ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:06:06.758567 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-1cd3ab19-64c4-4899-8891-ee6018c512ed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 05:06:06.758567 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:06.758567 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:06.758567 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:06.758567 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:06.758567 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:06.758567 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:06.758567 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:06.758567 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:06.758567 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:06.758567 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:06.758567 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:06.758567 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:06:06.758567 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:06:06.758567 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:06:06.758567 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:06:06.759432 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:06:06.759432 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:06:06.759432 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:06:06.759432 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:06:06.759432 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:06.759432 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:06.759432 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:06.759432 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:06.759432 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:06.759432 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:06.759432 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:06.759432 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:06.759432 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:06.759432 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:06.759432 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:06.759432 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:06.760806 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:06.760806 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:06.760806 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:06.760806 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:06.760806 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:06:06.760806 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:06:06.760806 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:06:06.760806 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:06:06.760806 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:06:06.760806 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:06:06.760806 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 05:06:06.760806 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:06.760806 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 236/486] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 05:06:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:06.766354 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3ab19-64c4-4899-8891-ee6018c512ed None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:06.802736 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff918b-55ba-4373-8c77-9cb7cd8628bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:06.803351 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff918b-55ba-4373-8c77-9cb7cd8628bb None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:06.803627 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff918b-55ba-4373-8c77-9cb7cd8628bb None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:06.803889 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff918b-55ba-4373-8c77-9cb7cd8628bb None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:06.804565 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deff918b-55ba-4373-8c77-9cb7cd8628bb None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:06.804861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deff918b-55ba-4373-8c77-9cb7cd8628bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:06.806280 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deff918b-55ba-4373-8c77-9cb7cd8628bb None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:06.825449 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 251/487] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Aug 31 05:06:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:06:06.830585 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff918b-55ba-4373-8c77-9cb7cd8628bb None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:06.847756 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:06.900586 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3ab19-64c4-4899-8891-ee6018c512ed None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:06.946515 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ab806c-5605-478b-9b97-e0fa297e92f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:06.947506 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab806c-5605-478b-9b97-e0fa297e92f6 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:06.947793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab806c-5605-478b-9b97-e0fa297e92f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:06.948066 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab806c-5605-478b-9b97-e0fa297e92f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:06.948739 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ab806c-5605-478b-9b97-e0fa297e92f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:06.949041 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ab806c-5605-478b-9b97-e0fa297e92f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:06.950996 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ab806c-5605-478b-9b97-e0fa297e92f6 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:06.962511 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 237/488] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Aug 31 05:06:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:06.962688 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff918b-55ba-4373-8c77-9cb7cd8628bb None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:06.963744 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63209060-389f-49d7-abfb-e8e1f6323879 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:06.964095 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63209060-389f-49d7-abfb-e8e1f6323879 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:06.964553 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63209060-389f-49d7-abfb-e8e1f6323879 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:06.970458 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 238/489] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:06.971627 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22ae70-5e84-4e97-8adc-d440073aa53a None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:06.971906 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22ae70-5e84-4e97-8adc-d440073aa53a None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:06.972171 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22ae70-5e84-4e97-8adc-d440073aa53a None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:06.973509 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 239/490] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:06.974249 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b22ae70-5e84-4e97-8adc-d440073aa53a None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:06.994862 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:07.002899 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe5a996-032d-47f5-9a33-29429e6776e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:07.004093 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5a996-032d-47f5-9a33-29429e6776e4 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:07.004762 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5a996-032d-47f5-9a33-29429e6776e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:07.005207 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5a996-032d-47f5-9a33-29429e6776e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:07.022065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe5a996-032d-47f5-9a33-29429e6776e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:07.022065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe5a996-032d-47f5-9a33-29429e6776e4 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:07.037108 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-ffe5a996-032d-47f5-9a33-29429e6776e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:07.037108 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:07.037108 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:07.037108 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:07.037108 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:07.037108 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:07.037108 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:07.037108 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:07.037108 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:07.037108 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:07.037108 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:07.037108 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:07.037108 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:07.037108 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:07.037108 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:07.037108 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:07.038046 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:07.038046 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:07.038046 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:07.038046 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:07.038046 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:07.038046 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:07.038046 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:07.038046 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:07.038046 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:07.038046 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:07.038046 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:07.038046 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:07.038046 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:07.038046 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:07.038046 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:07.038046 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:07.038046 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:07.039347 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:07.039347 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:07.039347 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:07.039347 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 252/491] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:06:07.039752 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7630bdb-e40c-4175-9e93-249c570998de None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:07.040035 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7630bdb-e40c-4175-9e93-249c570998de None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:07.040302 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7630bdb-e40c-4175-9e93-249c570998de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:07.058446 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f7630bdb-e40c-4175-9e93-249c570998de None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:07.059644 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f7630bdb-e40c-4175-9e93-249c570998de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:07.086751 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b22ae70-5e84-4e97-8adc-d440073aa53a None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:07.114726 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:07.123110 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59edfc2-c762-427a-90a2-d5b9a9fb100b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:07.123761 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59edfc2-c762-427a-90a2-d5b9a9fb100b None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:07.124035 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59edfc2-c762-427a-90a2-d5b9a9fb100b None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:07.124298 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59edfc2-c762-427a-90a2-d5b9a9fb100b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:07.138034 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 253/492] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:06:07.138034 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0446c-59b6-46e0-85de-e4ba835b8bd4 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:07.138034 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0446c-59b6-46e0-85de-e4ba835b8bd4 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:07.138034 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0446c-59b6-46e0-85de-e4ba835b8bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:07.143813 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59edfc2-c762-427a-90a2-d5b9a9fb100b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:07.145387 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59edfc2-c762-427a-90a2-d5b9a9fb100b None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:07.155620 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-38c0446c-59b6-46e0-85de-e4ba835b8bd4 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:07.156714 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-38c0446c-59b6-46e0-85de-e4ba835b8bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:07.161195 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-a59edfc2-c762-427a-90a2-d5b9a9fb100b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:07.161195 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:07.161195 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:07.161195 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:07.161195 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:07.161195 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:07.161195 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:07.161195 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:07.161195 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:07.161195 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:07.161195 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:07.161195 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:07.161195 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:07.161195 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:07.161195 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:07.161195 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:07.162113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:07.162113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:07.162113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:07.162113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:07.162113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:07.162113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:07.162113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:07.162113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:07.162113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:07.162113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:07.162113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:07.162113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:07.162113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:07.162113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:07.162113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:07.162113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:07.162113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:07.163061 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:07.163061 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:07.163061 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:07.163061 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 240/493] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:07.163061 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59edfc2-c762-427a-90a2-d5b9a9fb100b None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:07.240095 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:07.247828 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 254/494] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:06:07.248718 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0446c-59b6-46e0-85de-e4ba835b8bd4 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:07.292036 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59edfc2-c762-427a-90a2-d5b9a9fb100b None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:07.352146 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517e1e51-68d5-41fb-b3f4-f9712c705aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:07.352742 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e1e51-68d5-41fb-b3f4-f9712c705aa2 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:07.353013 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e1e51-68d5-41fb-b3f4-f9712c705aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:07.353276 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e1e51-68d5-41fb-b3f4-f9712c705aa2 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:07.353917 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517e1e51-68d5-41fb-b3f4-f9712c705aa2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:07.354236 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517e1e51-68d5-41fb-b3f4-f9712c705aa2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:07.355628 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517e1e51-68d5-41fb-b3f4-f9712c705aa2 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:07.378380 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 241/495] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:07.383937 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517e1e51-68d5-41fb-b3f4-f9712c705aa2 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:07.444061 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0446c-59b6-46e0-85de-e4ba835b8bd4 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:07.463679 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:07.482691 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716862b1-e60e-44ec-9962-b19353b5bc3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:07.482999 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716862b1-e60e-44ec-9962-b19353b5bc3a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:07.483258 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716862b1-e60e-44ec-9962-b19353b5bc3a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:07.483516 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716862b1-e60e-44ec-9962-b19353b5bc3a None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:07.484154 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716862b1-e60e-44ec-9962-b19353b5bc3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:07.484448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716862b1-e60e-44ec-9962-b19353b5bc3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:07.485819 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716862b1-e60e-44ec-9962-b19353b5bc3a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:07.497988 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 255/496] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:07.498997 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716862b1-e60e-44ec-9962-b19353b5bc3a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:07.522640 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517e1e51-68d5-41fb-b3f4-f9712c705aa2 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:07.559637 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:07.568663 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd956082-37a3-499b-8b2a-06d81ebb44b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:07.569294 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd956082-37a3-499b-8b2a-06d81ebb44b6 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:07.569568 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd956082-37a3-499b-8b2a-06d81ebb44b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:07.569834 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd956082-37a3-499b-8b2a-06d81ebb44b6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:07.577189 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd956082-37a3-499b-8b2a-06d81ebb44b6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:07.578693 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd956082-37a3-499b-8b2a-06d81ebb44b6 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:07.591344 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-dd956082-37a3-499b-8b2a-06d81ebb44b6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 05:06:07.591344 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:07.591344 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:07.591344 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:07.591344 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:07.591344 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:07.591344 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:07.591344 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:07.591344 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:07.591344 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:07.591344 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:07.591344 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:07.591344 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 05:06:07.591344 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 05:06:07.591344 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 05:06:07.591344 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 05:06:07.592337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:07.592337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:07.592337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 05:06:07.592337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:07.592337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 05:06:07.592337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:07.592337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:07.592337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:07.592337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:07.592337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:07.592337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:07.592337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:07.592337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:07.592337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:07.592337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:07.592337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:07.592337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:07.592337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:07.598217 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:07.598217 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:07.598217 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 05:06:07.598217 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 05:06:07.598217 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 05:06:07.598217 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 05:06:07.598217 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 05:06:07.598217 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 05:06:07.598217 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 05:06:07.598217 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:07.598217 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 242/497] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Aug 31 05:06:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:06:07.598217 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd956082-37a3-499b-8b2a-06d81ebb44b6 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:07.606453 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716862b1-e60e-44ec-9962-b19353b5bc3a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:07.641460 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dad5b8-b4b6-48c5-9d1b-65ca7f86e55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:07.642118 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dad5b8-b4b6-48c5-9d1b-65ca7f86e55a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:07.642400 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dad5b8-b4b6-48c5-9d1b-65ca7f86e55a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:07.642672 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dad5b8-b4b6-48c5-9d1b-65ca7f86e55a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:07.653242 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dad5b8-b4b6-48c5-9d1b-65ca7f86e55a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:06:07.659763 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-94dad5b8-b4b6-48c5-9d1b-65ca7f86e55a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 05:06:07.659763 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:07.659763 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:07.659763 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:07.659763 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:07.659763 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:07.659763 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:07.659763 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:07.659763 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:07.659763 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:07.659763 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:07.659763 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:07.659763 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:06:07.659763 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:06:07.659763 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:06:07.659763 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:06:07.660668 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:06:07.660668 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:06:07.660668 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:06:07.660668 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:06:07.660668 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:07.660668 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:07.660668 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:07.660668 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:07.660668 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:07.660668 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:07.660668 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:07.660668 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:07.660668 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:07.660668 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:07.660668 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:07.660668 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:07.661896 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:07.661896 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:07.661896 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:07.661896 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:07.661896 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:06:07.661896 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:06:07.661896 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:06:07.661896 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:06:07.661896 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:06:07.661896 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:06:07.661896 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 05:06:07.661896 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:07.661896 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 256/498] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 05:06:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:07.662815 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dad5b8-b4b6-48c5-9d1b-65ca7f86e55a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:07.711608 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:07.711775 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd956082-37a3-499b-8b2a-06d81ebb44b6 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:07.747534 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16717e06-1daa-48f4-9872-46334abd289a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:07.748159 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16717e06-1daa-48f4-9872-46334abd289a None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:07.748537 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16717e06-1daa-48f4-9872-46334abd289a None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:07.748804 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16717e06-1daa-48f4-9872-46334abd289a None admin] PATH_INFO: `/v3/users/glance` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:07.756593 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16717e06-1daa-48f4-9872-46334abd289a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:07.758108 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16717e06-1daa-48f4-9872-46334abd289a None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:07.769355 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-16717e06-1daa-48f4-9872-46334abd289a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 05:06:07.769355 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:07.769355 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:07.769355 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:07.769355 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:07.769355 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:07.769355 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:07.769355 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:07.769355 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:07.769355 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:07.769355 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:07.769355 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:07.769355 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 05:06:07.769355 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 05:06:07.769355 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 05:06:07.769355 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 05:06:07.770291 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:07.770291 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:07.770291 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 05:06:07.770291 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:07.770291 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 05:06:07.770291 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:07.770291 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:07.770291 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:07.770291 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:07.770291 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:07.770291 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:07.770291 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:07.770291 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:07.770291 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:07.770291 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:07.770291 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:07.770291 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:07.770291 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:07.771200 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:07.771200 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:07.771200 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 05:06:07.771200 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 05:06:07.771200 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 05:06:07.771200 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 05:06:07.771200 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 05:06:07.771200 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 05:06:07.771200 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 05:06:07.771200 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:07.771200 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 243/499] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:06:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:07.771200 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16717e06-1daa-48f4-9872-46334abd289a None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:07.781240 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dad5b8-b4b6-48c5-9d1b-65ca7f86e55a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:07.817072 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b4f36-96d0-4640-b50b-d4b7bf4dc930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:07.817695 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b4f36-96d0-4640-b50b-d4b7bf4dc930 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:07.817968 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b4f36-96d0-4640-b50b-d4b7bf4dc930 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:07.818241 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b4f36-96d0-4640-b50b-d4b7bf4dc930 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:07.821261 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:07.829732 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248b4f36-96d0-4640-b50b-d4b7bf4dc930 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:06:07.835722 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-248b4f36-96d0-4640-b50b-d4b7bf4dc930 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 05:06:07.835722 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:07.835722 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:07.835722 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:07.835722 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:07.835722 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:07.835722 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:07.835722 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:07.835722 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:07.835722 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:07.835722 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:07.835722 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:07.835722 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:06:07.835722 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:06:07.835722 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:06:07.835722 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:06:07.836626 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:06:07.836626 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:06:07.836626 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:06:07.836626 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:06:07.836626 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:07.836626 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:07.836626 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:07.836626 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:07.836626 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:07.836626 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:07.836626 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:07.836626 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:07.836626 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:07.836626 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:07.836626 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:07.836626 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:07.837458 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:07.837458 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:07.837458 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:07.837458 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:07.837458 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:06:07.837458 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:06:07.837458 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:06:07.837458 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:06:07.837458 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:06:07.837458 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:06:07.837458 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 05:06:07.837458 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:07.837458 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 257/500] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 05:06:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:07.838169 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b4f36-96d0-4640-b50b-d4b7bf4dc930 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:07.881569 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16717e06-1daa-48f4-9872-46334abd289a None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:07.915133 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a2bca-3fce-4325-933d-a04b600732aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:07.915735 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a2bca-3fce-4325-933d-a04b600732aa None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:07.916008 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a2bca-3fce-4325-933d-a04b600732aa None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:07.916322 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a2bca-3fce-4325-933d-a04b600732aa None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:07.917058 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7a2bca-3fce-4325-933d-a04b600732aa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:07.917357 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7a2bca-3fce-4325-933d-a04b600732aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:07.918847 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7a2bca-3fce-4325-933d-a04b600732aa None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:07.939305 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:07.943560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b4f36-96d0-4640-b50b-d4b7bf4dc930 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:07.955716 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 244/501] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Aug 31 05:06:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:07.956675 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a2bca-3fce-4325-933d-a04b600732aa None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:07.985099 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae85c11c-553e-4858-8e6a-9b9f7305b690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:07.985597 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85c11c-553e-4858-8e6a-9b9f7305b690 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:07.985885 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85c11c-553e-4858-8e6a-9b9f7305b690 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:07.986180 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85c11c-553e-4858-8e6a-9b9f7305b690 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:07.986819 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae85c11c-553e-4858-8e6a-9b9f7305b690 None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:07.987125 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae85c11c-553e-4858-8e6a-9b9f7305b690 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:07.988836 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae85c11c-553e-4858-8e6a-9b9f7305b690 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:07.997085 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 258/502] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Aug 31 05:06:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:07.997862 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae85c11c-553e-4858-8e6a-9b9f7305b690 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:08.060090 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a2bca-3fce-4325-933d-a04b600732aa None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:08.067275 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:08.094383 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d120bc94-4ba9-43f3-bbcb-f216c474825f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:08.094977 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d120bc94-4ba9-43f3-bbcb-f216c474825f None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:08.095248 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d120bc94-4ba9-43f3-bbcb-f216c474825f None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:08.095511 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d120bc94-4ba9-43f3-bbcb-f216c474825f None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:08.095599 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae85c11c-553e-4858-8e6a-9b9f7305b690 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:08.096225 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d120bc94-4ba9-43f3-bbcb-f216c474825f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:08.096521 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d120bc94-4ba9-43f3-bbcb-f216c474825f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:08.097968 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d120bc94-4ba9-43f3-bbcb-f216c474825f None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:08.114608 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 245/503] 10.222.0.36 () {56 vars in 1137 bytes} [Wed Aug 31 05:06:07 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:08.115472 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d120bc94-4ba9-43f3-bbcb-f216c474825f None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:08.134024 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35995af2-5fdf-4e67-b098-b19584c8baa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:08.134619 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35995af2-5fdf-4e67-b098-b19584c8baa0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:08.134892 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35995af2-5fdf-4e67-b098-b19584c8baa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:08.135154 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35995af2-5fdf-4e67-b098-b19584c8baa0 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:08.135801 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35995af2-5fdf-4e67-b098-b19584c8baa0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:08.136094 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35995af2-5fdf-4e67-b098-b19584c8baa0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:08.137452 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35995af2-5fdf-4e67-b098-b19584c8baa0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:08.145741 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 259/504] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Aug 31 05:06:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:08.146902 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60272c0-22d2-4145-81d9-1dd41d157c95 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:08.147175 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60272c0-22d2-4145-81d9-1dd41d157c95 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:08.147431 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60272c0-22d2-4145-81d9-1dd41d157c95 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:08.152598 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 260/505] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:08.152598 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60272c0-22d2-4145-81d9-1dd41d157c95 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:08.173084 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:08.216332 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d120bc94-4ba9-43f3-bbcb-f216c474825f None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:08.248590 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60272c0-22d2-4145-81d9-1dd41d157c95 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:08.249834 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54898d73-febd-4e96-af06-cd8994dfa759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:08.250561 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54898d73-febd-4e96-af06-cd8994dfa759 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:08.250820 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54898d73-febd-4e96-af06-cd8994dfa759 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:08.251086 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54898d73-febd-4e96-af06-cd8994dfa759 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:08.280781 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:08.285553 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e6ecad-553d-46a4-af29-581a9fae7dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:08.286222 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6ecad-553d-46a4-af29-581a9fae7dab None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:08.286503 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6ecad-553d-46a4-af29-581a9fae7dab None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:08.286772 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6ecad-553d-46a4-af29-581a9fae7dab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:08.297535 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e6ecad-553d-46a4-af29-581a9fae7dab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:08.299034 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e6ecad-553d-46a4-af29-581a9fae7dab None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:08.312924 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-93e6ecad-553d-46a4-af29-581a9fae7dab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:08.312924 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:08.312924 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:08.312924 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:08.312924 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:08.312924 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:08.312924 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:08.312924 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:08.312924 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:08.312924 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:08.312924 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:08.312924 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:08.312924 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:08.312924 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:08.312924 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:08.312924 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:08.313941 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:08.313941 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:08.313941 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:08.313941 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:08.313941 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:08.313941 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:08.313941 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:08.313941 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:08.313941 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:08.313941 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:08.313941 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:08.313941 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:08.313941 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:08.313941 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:08.313941 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:08.313941 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:08.313941 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:08.314844 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:08.314844 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:08.314844 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:08.314844 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 261/506] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:08.314844 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e6ecad-553d-46a4-af29-581a9fae7dab None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:08.322608 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54898d73-febd-4e96-af06-cd8994dfa759 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:08.324082 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54898d73-febd-4e96-af06-cd8994dfa759 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:08.388967 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:08.414484 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e6ecad-553d-46a4-af29-581a9fae7dab None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:08.423278 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 246/507] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:06:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 05:06:08.424274 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54898d73-febd-4e96-af06-cd8994dfa759 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:08.450489 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ca401-32b2-454b-b8d1-834ab779d592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:08.451087 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ca401-32b2-454b-b8d1-834ab779d592 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:08.451404 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ca401-32b2-454b-b8d1-834ab779d592 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:08.451674 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ca401-32b2-454b-b8d1-834ab779d592 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:08.483877 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:08.518037 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08ca401-32b2-454b-b8d1-834ab779d592 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:08.519420 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08ca401-32b2-454b-b8d1-834ab779d592 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:08.522272 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54898d73-febd-4e96-af06-cd8994dfa759 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:08.556320 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa0d1c-fce0-416e-a820-fcd95c374568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:08.556947 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa0d1c-fce0-416e-a820-fcd95c374568 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:08.557224 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa0d1c-fce0-416e-a820-fcd95c374568 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:08.557494 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa0d1c-fce0-416e-a820-fcd95c374568 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:08.567350 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa0d1c-fce0-416e-a820-fcd95c374568 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:08.568797 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa0d1c-fce0-416e-a820-fcd95c374568 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:08.583087 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-63fa0d1c-fce0-416e-a820-fcd95c374568 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:08.583087 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:08.583087 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:08.583087 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:08.583087 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:08.583087 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:08.583087 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:08.583087 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:08.583087 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:08.583087 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:08.583087 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:08.583087 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:08.583087 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:08.583087 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:08.583087 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:08.583087 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:08.583993 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:08.583993 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:08.583993 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:08.583993 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:08.583993 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:08.583993 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:08.583993 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:08.583993 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:08.583993 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:08.583993 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:08.583993 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:08.583993 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:08.583993 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:08.583993 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:08.583993 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:08.583993 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:08.583993 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:08.584843 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:08.584843 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:08.584843 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:08.584843 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 247/508] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:08.585399 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdabfeb-3cc0-4955-9285-75e49fccfa57 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:08.585682 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdabfeb-3cc0-4955-9285-75e49fccfa57 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:08.585947 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdabfeb-3cc0-4955-9285-75e49fccfa57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:08.594314 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:08.603514 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-dbdabfeb-3cc0-4955-9285-75e49fccfa57 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:08.604685 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-dbdabfeb-3cc0-4955-9285-75e49fccfa57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:08.621687 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 262/509] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:06:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 05:06:08.622747 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ca401-32b2-454b-b8d1-834ab779d592 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:08.673407 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 248/510] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:06:08.674870 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdabfeb-3cc0-4955-9285-75e49fccfa57 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:08.720139 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:08.721536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ca401-32b2-454b-b8d1-834ab779d592 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:08.755724 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c3735-fd35-4b9c-ace0-2dc13a8df542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:08.756326 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c3735-fd35-4b9c-ace0-2dc13a8df542 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:08.756598 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c3735-fd35-4b9c-ace0-2dc13a8df542 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:08.756859 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c3735-fd35-4b9c-ace0-2dc13a8df542 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:08.757491 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c3735-fd35-4b9c-ace0-2dc13a8df542 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:08.757783 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c3735-fd35-4b9c-ace0-2dc13a8df542 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:08.759168 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c3735-fd35-4b9c-ace0-2dc13a8df542 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:08.771418 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 263/511] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:08.772063 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c3735-fd35-4b9c-ace0-2dc13a8df542 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:08.779058 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdabfeb-3cc0-4955-9285-75e49fccfa57 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:08.804583 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:08.812690 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97965ad2-86e7-4494-9e90-efb28563266f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:08.813321 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97965ad2-86e7-4494-9e90-efb28563266f None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:08.813594 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97965ad2-86e7-4494-9e90-efb28563266f None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:08.813857 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97965ad2-86e7-4494-9e90-efb28563266f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:08.823639 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97965ad2-86e7-4494-9e90-efb28563266f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:08.825077 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97965ad2-86e7-4494-9e90-efb28563266f None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:08.840116 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-97965ad2-86e7-4494-9e90-efb28563266f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:08.840116 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:08.840116 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:08.840116 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:08.840116 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:08.840116 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:08.840116 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:08.840116 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:08.840116 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:08.840116 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:08.840116 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:08.840116 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:08.840116 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:08.840116 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:08.840116 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:08.840116 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:08.840998 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:08.840998 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:08.840998 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:08.840998 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:08.840998 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:08.840998 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:08.840998 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:08.840998 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:08.840998 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:08.840998 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:08.840998 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:08.840998 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:08.840998 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:08.840998 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:08.840998 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:08.840998 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:08.840998 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:08.842676 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:08.842676 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:08.842676 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:08.842676 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 249/512] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:08.842676 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97965ad2-86e7-4494-9e90-efb28563266f None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:08.877645 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c3735-fd35-4b9c-ace0-2dc13a8df542 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:08.915264 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e094c3b7-98e8-45a6-8b2b-e38f9542240f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:08.915862 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e094c3b7-98e8-45a6-8b2b-e38f9542240f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:08.916199 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e094c3b7-98e8-45a6-8b2b-e38f9542240f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:08.916459 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e094c3b7-98e8-45a6-8b2b-e38f9542240f None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:08.917795 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e094c3b7-98e8-45a6-8b2b-e38f9542240f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:08.917795 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e094c3b7-98e8-45a6-8b2b-e38f9542240f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:08.918765 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e094c3b7-98e8-45a6-8b2b-e38f9542240f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:08.934745 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 264/513] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:08.934745 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e094c3b7-98e8-45a6-8b2b-e38f9542240f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:08.950269 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:08.954497 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97965ad2-86e7-4494-9e90-efb28563266f None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:08.987710 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4479195-b464-43ff-b19c-571327c16b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:08.988306 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4479195-b464-43ff-b19c-571327c16b32 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:08.988586 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4479195-b464-43ff-b19c-571327c16b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:08.988848 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4479195-b464-43ff-b19c-571327c16b32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:08.999516 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4479195-b464-43ff-b19c-571327c16b32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:09.001019 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4479195-b464-43ff-b19c-571327c16b32 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:09.015327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-f4479195-b464-43ff-b19c-571327c16b32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:09.015327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:09.015327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:09.015327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:09.015327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:09.015327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:09.015327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:09.015327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:09.015327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:09.015327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:09.015327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:09.015327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:09.015327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:09.015327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:09.015327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:09.015327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:09.016273 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:09.016273 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:09.016273 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:09.016273 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:09.016273 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:09.016273 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:09.016273 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:09.016273 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:09.016273 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:09.016273 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:09.016273 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:09.016273 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:09.016273 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:09.016273 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:09.016273 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:09.016273 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:09.016273 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:09.017144 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:09.017144 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:09.017144 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:09.017144 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 250/514] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:09.017144 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4479195-b464-43ff-b19c-571327c16b32 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:09.028710 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:09.036340 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e094c3b7-98e8-45a6-8b2b-e38f9542240f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:09.069022 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc6e61-7676-4729-b651-72aed09c682a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:09.069633 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc6e61-7676-4729-b651-72aed09c682a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:09.069904 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc6e61-7676-4729-b651-72aed09c682a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:09.070196 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc6e61-7676-4729-b651-72aed09c682a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:09.077005 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc6e61-7676-4729-b651-72aed09c682a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:09.078478 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc6e61-7676-4729-b651-72aed09c682a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:09.090928 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-57bc6e61-7676-4729-b651-72aed09c682a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 05:06:09.090928 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:09.090928 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:09.090928 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:09.090928 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:09.090928 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:09.090928 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:09.090928 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:09.090928 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:09.090928 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:09.090928 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:09.090928 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:09.090928 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 05:06:09.090928 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 05:06:09.090928 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 05:06:09.090928 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 05:06:09.091925 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:09.091925 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:09.091925 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 05:06:09.091925 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:09.091925 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 05:06:09.091925 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:09.091925 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:09.091925 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:09.091925 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:09.091925 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:09.091925 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:09.091925 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:09.091925 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:09.091925 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:09.091925 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:09.091925 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:09.091925 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:09.091925 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:09.092877 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:09.092877 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:09.092877 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 05:06:09.092877 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 05:06:09.092877 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 05:06:09.092877 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 05:06:09.092877 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 05:06:09.092877 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 05:06:09.092877 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 05:06:09.092877 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:09.092877 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 265/515] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:06:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:09.092877 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc6e61-7676-4729-b651-72aed09c682a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:09.115886 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4479195-b464-43ff-b19c-571327c16b32 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:09.148908 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02368b08-2620-48fb-b1b6-628ee9544f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:09.149484 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02368b08-2620-48fb-b1b6-628ee9544f8d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:09.149758 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02368b08-2620-48fb-b1b6-628ee9544f8d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:09.150051 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02368b08-2620-48fb-b1b6-628ee9544f8d None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:09.150695 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02368b08-2620-48fb-b1b6-628ee9544f8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:09.150991 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02368b08-2620-48fb-b1b6-628ee9544f8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:09.152365 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02368b08-2620-48fb-b1b6-628ee9544f8d None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:09.164031 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 251/516] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:09.164703 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02368b08-2620-48fb-b1b6-628ee9544f8d None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:09.176201 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:09.197973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc6e61-7676-4729-b651-72aed09c682a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:09.230413 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f34d0-ceec-4ac9-8ec5-ed5c35e4f15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:09.231039 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f34d0-ceec-4ac9-8ec5-ed5c35e4f15f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:09.231302 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f34d0-ceec-4ac9-8ec5-ed5c35e4f15f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:09.231564 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f34d0-ceec-4ac9-8ec5-ed5c35e4f15f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:09.238895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5f34d0-ceec-4ac9-8ec5-ed5c35e4f15f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:09.240315 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5f34d0-ceec-4ac9-8ec5-ed5c35e4f15f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:09.251536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-1f5f34d0-ceec-4ac9-8ec5-ed5c35e4f15f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 05:06:09.251536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:09.251536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:09.251536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:09.251536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:09.251536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:09.251536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:09.251536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:09.251536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:09.251536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:09.251536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:09.251536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:09.251536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 05:06:09.251536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 05:06:09.251536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 05:06:09.251536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 05:06:09.252448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:09.252448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:09.252448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 05:06:09.252448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:09.252448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 05:06:09.252448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:09.252448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:09.252448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:09.252448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:09.252448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:09.252448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:09.252448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:09.252448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:09.252448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:09.252448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:09.252448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:09.252448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:09.252448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:09.253331 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:09.253331 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:09.253331 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 05:06:09.253331 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 05:06:09.253331 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 05:06:09.253331 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 05:06:09.253331 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 05:06:09.253331 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 05:06:09.253331 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 05:06:09.253331 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:09.253331 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 266/517] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 05:06:09 2022] GET /identity/v3/users/neutron => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:09.253331 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f34d0-ceec-4ac9-8ec5-ed5c35e4f15f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:09.266715 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02368b08-2620-48fb-b1b6-628ee9544f8d None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:09.270157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:09.301053 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2087f9-b045-475e-9b64-c5c655d8711d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:09.301956 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2087f9-b045-475e-9b64-c5c655d8711d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:09.302275 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2087f9-b045-475e-9b64-c5c655d8711d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:09.302577 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2087f9-b045-475e-9b64-c5c655d8711d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:09.315033 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2087f9-b045-475e-9b64-c5c655d8711d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:09.316569 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2087f9-b045-475e-9b64-c5c655d8711d None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:09.332583 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-ce2087f9-b045-475e-9b64-c5c655d8711d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:09.332583 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:09.332583 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:09.332583 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:09.332583 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:09.332583 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:09.332583 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:09.332583 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:09.332583 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:09.332583 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:09.332583 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:09.332583 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:09.332583 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:09.332583 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:09.332583 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:09.332583 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:09.333494 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:09.333494 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:09.333494 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:09.333494 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:09.333494 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:09.333494 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:09.333494 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:09.333494 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:09.333494 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:09.333494 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:09.333494 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:09.333494 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:09.333494 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:09.333494 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:09.333494 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:09.333494 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:09.333494 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:09.334374 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:09.334374 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:09.334374 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:09.334374 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 252/518] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:09.334883 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2087f9-b045-475e-9b64-c5c655d8711d None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:09.353308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f34d0-ceec-4ac9-8ec5-ed5c35e4f15f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:09.387462 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d40287-fde6-4078-a972-cf7f38fdea24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:09.388164 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d40287-fde6-4078-a972-cf7f38fdea24 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:09.388438 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d40287-fde6-4078-a972-cf7f38fdea24 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:09.388705 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d40287-fde6-4078-a972-cf7f38fdea24 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:09.389394 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d40287-fde6-4078-a972-cf7f38fdea24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:09.389688 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d40287-fde6-4078-a972-cf7f38fdea24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:09.391083 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d40287-fde6-4078-a972-cf7f38fdea24 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:09.402190 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:09.402811 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 267/519] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:09.403395 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d40287-fde6-4078-a972-cf7f38fdea24 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:09.442865 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2087f9-b045-475e-9b64-c5c655d8711d None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:09.474999 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e386ce-959c-4bf4-b5e3-78341c6c635f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:09.475601 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e386ce-959c-4bf4-b5e3-78341c6c635f None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:09.475871 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e386ce-959c-4bf4-b5e3-78341c6c635f None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:09.476136 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e386ce-959c-4bf4-b5e3-78341c6c635f None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:09.476857 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e386ce-959c-4bf4-b5e3-78341c6c635f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:09.477154 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e386ce-959c-4bf4-b5e3-78341c6c635f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:09.478628 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e386ce-959c-4bf4-b5e3-78341c6c635f None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:09.483370 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:09.495581 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 253/520] 10.222.0.36 () {56 vars in 1137 bytes} [Wed Aug 31 05:06:09 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:09.496385 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e386ce-959c-4bf4-b5e3-78341c6c635f None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:09.508503 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d40287-fde6-4078-a972-cf7f38fdea24 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:09.543375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59333a4-d6ad-4eda-b1c2-21de0adcaaa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:09.543996 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59333a4-d6ad-4eda-b1c2-21de0adcaaa6 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:09.544268 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59333a4-d6ad-4eda-b1c2-21de0adcaaa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:09.544531 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59333a4-d6ad-4eda-b1c2-21de0adcaaa6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:09.552175 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59333a4-d6ad-4eda-b1c2-21de0adcaaa6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:09.553549 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59333a4-d6ad-4eda-b1c2-21de0adcaaa6 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:09.565376 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-b59333a4-d6ad-4eda-b1c2-21de0adcaaa6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:09.565376 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:09.565376 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:09.565376 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:09.565376 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:09.565376 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:09.565376 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:09.565376 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:09.565376 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:09.565376 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:09.565376 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:09.565376 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:09.565376 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 05:06:09.565376 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 05:06:09.565376 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 05:06:09.566337 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 05:06:09.566337 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:09.566337 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:09.566337 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:09.566337 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:09.566337 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:09.566337 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:09.566337 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:09.566337 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:09.566337 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:09.566337 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:09.566337 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:09.566337 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:09.566337 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:09.566337 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:09.566337 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:09.566337 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:09.567200 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 05:06:09.567200 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 05:06:09.567200 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 05:06:09.567200 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 05:06:09.567200 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 05:06:09.567200 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 05:06:09.567200 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:09.567200 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:09.567200 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 268/521] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:06:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:09.567200 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59333a4-d6ad-4eda-b1c2-21de0adcaaa6 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:09.602264 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e386ce-959c-4bf4-b5e3-78341c6c635f None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:09.622783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:09.637135 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57e876a-818d-415d-ab19-a5aa1d33103d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:09.637740 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e876a-818d-415d-ab19-a5aa1d33103d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:09.638027 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e876a-818d-415d-ab19-a5aa1d33103d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:09.638308 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e876a-818d-415d-ab19-a5aa1d33103d None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:09.639077 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57e876a-818d-415d-ab19-a5aa1d33103d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:09.639384 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57e876a-818d-415d-ab19-a5aa1d33103d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:09.640827 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57e876a-818d-415d-ab19-a5aa1d33103d None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:09.657826 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 254/522] 10.222.0.36 () {56 vars in 1139 bytes} [Wed Aug 31 05:06:09 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:09.659713 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57e876a-818d-415d-ab19-a5aa1d33103d None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:09.671202 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59333a4-d6ad-4eda-b1c2-21de0adcaaa6 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:09.704166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d2368-7e42-47c5-b89d-55286e0737bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:09.704769 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d2368-7e42-47c5-b89d-55286e0737bf None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:09.705038 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d2368-7e42-47c5-b89d-55286e0737bf None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:09.705298 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d2368-7e42-47c5-b89d-55286e0737bf None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:09.705927 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195d2368-7e42-47c5-b89d-55286e0737bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:09.706249 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195d2368-7e42-47c5-b89d-55286e0737bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:09.707603 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195d2368-7e42-47c5-b89d-55286e0737bf None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:09.708984 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:09.720004 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 269/523] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:09.720781 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d2368-7e42-47c5-b89d-55286e0737bf None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:09.761600 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57e876a-818d-415d-ab19-a5aa1d33103d None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:09.794385 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e7556-bf4d-4eb6-8cd2-0ee016933d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:09.795001 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e7556-bf4d-4eb6-8cd2-0ee016933d6e None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:09.795302 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e7556-bf4d-4eb6-8cd2-0ee016933d6e None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:09.795570 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e7556-bf4d-4eb6-8cd2-0ee016933d6e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:09.802329 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e7556-bf4d-4eb6-8cd2-0ee016933d6e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:09.803724 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e7556-bf4d-4eb6-8cd2-0ee016933d6e None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:09.815566 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-357e7556-bf4d-4eb6-8cd2-0ee016933d6e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:09.815566 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:09.815566 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:09.815566 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:09.815566 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:09.815566 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:09.815566 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:09.815566 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:09.815566 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:09.815566 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:09.815566 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:09.815566 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:09.815566 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 05:06:09.815566 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 05:06:09.815566 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 05:06:09.816467 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 05:06:09.816467 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:09.816467 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:09.816467 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:09.816467 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:09.816467 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:09.816467 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:09.816467 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:09.816467 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:09.816467 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:09.816467 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:09.816467 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:09.816467 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:09.816467 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:09.816467 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:09.816467 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:09.816467 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:09.817320 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 05:06:09.817320 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 05:06:09.817320 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 05:06:09.817320 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 05:06:09.817320 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 05:06:09.817320 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 05:06:09.817320 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:09.817320 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:09.817320 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 255/524] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:06:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:09.817776 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e7556-bf4d-4eb6-8cd2-0ee016933d6e None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:09.818384 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d2368-7e42-47c5-b89d-55286e0737bf None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:09.853614 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ad0ce-5dca-462e-8a55-17bf622e7287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:09.854259 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ad0ce-5dca-462e-8a55-17bf622e7287 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:09.854534 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ad0ce-5dca-462e-8a55-17bf622e7287 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:09.854804 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ad0ce-5dca-462e-8a55-17bf622e7287 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:09.859413 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:09.915333 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:09.918976 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e7556-bf4d-4eb6-8cd2-0ee016933d6e None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:09.923044 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4ad0ce-5dca-462e-8a55-17bf622e7287 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:09.924575 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4ad0ce-5dca-462e-8a55-17bf622e7287 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:09.952768 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73525af1-6d64-4e71-838b-1541b2e8f643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:09.953370 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73525af1-6d64-4e71-838b-1541b2e8f643 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:09.953643 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73525af1-6d64-4e71-838b-1541b2e8f643 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:09.953910 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73525af1-6d64-4e71-838b-1541b2e8f643 None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:09.954628 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73525af1-6d64-4e71-838b-1541b2e8f643 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:09.954928 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73525af1-6d64-4e71-838b-1541b2e8f643 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:09.956370 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73525af1-6d64-4e71-838b-1541b2e8f643 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:09.974762 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 256/525] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 05:06:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:09.975640 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73525af1-6d64-4e71-838b-1541b2e8f643 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:10.020516 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 270/526] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:06:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 05:06:10.021578 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ad0ce-5dca-462e-8a55-17bf622e7287 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:10.073772 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:10.077814 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73525af1-6d64-4e71-838b-1541b2e8f643 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:10.111276 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:10.111955 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e7a09-fd3a-4e2f-afd3-1b157f61aeba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:10.112550 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e7a09-fd3a-4e2f-afd3-1b157f61aeba None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:10.112819 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e7a09-fd3a-4e2f-afd3-1b157f61aeba None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:10.113083 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e7a09-fd3a-4e2f-afd3-1b157f61aeba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:10.118938 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ad0ce-5dca-462e-8a55-17bf622e7287 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:10.152816 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488eea2b-4b92-4756-a30f-cdf15da745a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:10.153411 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488eea2b-4b92-4756-a30f-cdf15da745a4 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:10.153684 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488eea2b-4b92-4756-a30f-cdf15da745a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:10.153946 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488eea2b-4b92-4756-a30f-cdf15da745a4 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:10.154640 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488eea2b-4b92-4756-a30f-cdf15da745a4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:10.156077 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488eea2b-4b92-4756-a30f-cdf15da745a4 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:10.176654 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 271/527] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Aug 31 05:06:10 2022] POST /identity/v3/users => generated 247 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:06:10.177439 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488eea2b-4b92-4756-a30f-cdf15da745a4 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:10.183067 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305e7a09-fd3a-4e2f-afd3-1b157f61aeba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:10.184893 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305e7a09-fd3a-4e2f-afd3-1b157f61aeba None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:10.283154 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488eea2b-4b92-4756-a30f-cdf15da745a4 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:10.317968 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:10.322431 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df682ef0-af84-43fa-8a6c-35cd7d94343c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:10.322871 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 257/528] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:06:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 05:06:10.323037 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df682ef0-af84-43fa-8a6c-35cd7d94343c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:10.323314 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df682ef0-af84-43fa-8a6c-35cd7d94343c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:10.323579 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df682ef0-af84-43fa-8a6c-35cd7d94343c None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:10.323669 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e7a09-fd3a-4e2f-afd3-1b157f61aeba None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:10.324440 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df682ef0-af84-43fa-8a6c-35cd7d94343c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:10.324689 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df682ef0-af84-43fa-8a6c-35cd7d94343c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:10.326167 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df682ef0-af84-43fa-8a6c-35cd7d94343c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:10.345677 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 272/529] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 05:06:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:10.346551 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df682ef0-af84-43fa-8a6c-35cd7d94343c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:10.358756 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:10.439023 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e7a09-fd3a-4e2f-afd3-1b157f61aeba None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:10.461740 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df682ef0-af84-43fa-8a6c-35cd7d94343c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:10.475044 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6b5056-3349-426c-ae11-cec2c6ba4148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:10.475666 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b5056-3349-426c-ae11-cec2c6ba4148 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:10.476011 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b5056-3349-426c-ae11-cec2c6ba4148 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:10.476469 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b5056-3349-426c-ae11-cec2c6ba4148 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:10.477199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6b5056-3349-426c-ae11-cec2c6ba4148 None admin] RBAC: Adding query filter params (role.id=207bf98e31774cc0b21249de6d96155b, scope.project.id=e01ce1b1282e44a39d659b87f120995e, user.id=f27665b1574a4db7b7a9771f7d775826) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:10.477565 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6b5056-3349-426c-ae11-cec2c6ba4148 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:10.479099 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6b5056-3349-426c-ae11-cec2c6ba4148 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:10.494104 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 258/530] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:06:10 2022] GET /identity/v3/role_assignments?user.id=f27665b1574a4db7b7a9771f7d775826&scope.project.id=e01ce1b1282e44a39d659b87f120995e&role.id=207bf98e31774cc0b21249de6d96155b => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:10.494104 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6b5056-3349-426c-ae11-cec2c6ba4148 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:10.509405 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe770a7-4c6b-47bb-a523-ac0a3777fb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:10.510304 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe770a7-4c6b-47bb-a523-ac0a3777fb93 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:10.510641 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe770a7-4c6b-47bb-a523-ac0a3777fb93 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:10.510909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe770a7-4c6b-47bb-a523-ac0a3777fb93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:10.522861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe770a7-4c6b-47bb-a523-ac0a3777fb93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:10.524743 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe770a7-4c6b-47bb-a523-ac0a3777fb93 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:10.541556 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-abe770a7-4c6b-47bb-a523-ac0a3777fb93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:10.541556 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:10.541556 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:10.541556 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:10.541556 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:10.541556 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:10.541556 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:10.541556 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:10.541556 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:10.541556 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:10.541556 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:10.541556 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:10.541556 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:10.541556 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:10.541556 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:10.541556 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:10.542777 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:10.542777 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:10.542777 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:10.542777 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:10.542777 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:10.542777 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:10.542777 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:10.542777 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:10.542777 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:10.542777 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:10.542777 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:10.542777 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:10.542777 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:10.542777 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:10.542777 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:10.542777 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:10.542777 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:10.543834 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:10.543834 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:10.543834 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:10.544563 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 273/531] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:10.545472 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe770a7-4c6b-47bb-a523-ac0a3777fb93 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:10.579903 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:10.622984 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:10.633748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6b5056-3349-426c-ae11-cec2c6ba4148 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:10.668115 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe770a7-4c6b-47bb-a523-ac0a3777fb93 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:10.686049 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d42ef-aa2e-4511-a115-01599790274b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:10.686049 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d42ef-aa2e-4511-a115-01599790274b None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:10.686049 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d42ef-aa2e-4511-a115-01599790274b None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:10.686049 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d42ef-aa2e-4511-a115-01599790274b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:10.701892 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51d42ef-aa2e-4511-a115-01599790274b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:10.704048 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51d42ef-aa2e-4511-a115-01599790274b None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:10.708594 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad8414-91b4-46e3-9f83-ad65af765580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:10.709277 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad8414-91b4-46e3-9f83-ad65af765580 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:10.709643 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad8414-91b4-46e3-9f83-ad65af765580 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:10.709974 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad8414-91b4-46e3-9f83-ad65af765580 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:10.710846 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad8414-91b4-46e3-9f83-ad65af765580 None admin] RBAC: Adding query filter params (role.id=207bf98e31774cc0b21249de6d96155b, scope.project.id=e01ce1b1282e44a39d659b87f120995e, user.id=c08e6024b0c8420aa274653c2a916edd) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:10.711217 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad8414-91b4-46e3-9f83-ad65af765580 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:10.712767 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad8414-91b4-46e3-9f83-ad65af765580 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:10.721661 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-c51d42ef-aa2e-4511-a115-01599790274b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:10.721661 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:10.721661 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:10.721661 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:10.721661 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:10.721661 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:10.721661 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:10.721661 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:10.721661 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:10.721661 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:10.721661 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:10.721661 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:10.721661 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:10.721661 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:10.721661 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:10.721661 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:10.722598 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:10.722598 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:10.722598 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:10.722598 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:10.722598 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:10.722598 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:10.722598 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:10.722598 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:10.722598 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:10.722598 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:10.722598 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:10.722598 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:10.722598 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:10.722598 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:10.722598 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:10.722598 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:10.722598 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:10.723444 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:10.723444 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:10.723444 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:10.729429 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 274/533] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:06:10 2022] GET /identity/v3/role_assignments?user.id=c08e6024b0c8420aa274653c2a916edd&scope.project.id=e01ce1b1282e44a39d659b87f120995e&role.id=207bf98e31774cc0b21249de6d96155b => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:10.729429 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad8414-91b4-46e3-9f83-ad65af765580 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:10.729882 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 259/533] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:10.731406 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d42ef-aa2e-4511-a115-01599790274b None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:10.854630 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d42ef-aa2e-4511-a115-01599790274b None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:10.872215 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:10.904894 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbfe0df-ecce-40ec-8dd6-5a20c4386677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:10.905932 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfe0df-ecce-40ec-8dd6-5a20c4386677 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:10.906399 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfe0df-ecce-40ec-8dd6-5a20c4386677 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:10.906799 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfe0df-ecce-40ec-8dd6-5a20c4386677 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:10.908103 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbfe0df-ecce-40ec-8dd6-5a20c4386677 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:10.908680 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbfe0df-ecce-40ec-8dd6-5a20c4386677 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:10.910498 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbfe0df-ecce-40ec-8dd6-5a20c4386677 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:10.933885 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 260/534] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:10.940556 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbfe0df-ecce-40ec-8dd6-5a20c4386677 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:10.965626 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad8414-91b4-46e3-9f83-ad65af765580 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:11.005327 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:11.017046 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2367e122-badd-49f0-be2e-8fb0a4ae633d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:11.017894 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2367e122-badd-49f0-be2e-8fb0a4ae633d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:11.018337 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2367e122-badd-49f0-be2e-8fb0a4ae633d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:11.019244 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2367e122-badd-49f0-be2e-8fb0a4ae633d None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:11.020067 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2367e122-badd-49f0-be2e-8fb0a4ae633d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:11.020366 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2367e122-badd-49f0-be2e-8fb0a4ae633d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:11.021751 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2367e122-badd-49f0-be2e-8fb0a4ae633d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:11.037167 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 275/535] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:11.042461 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2367e122-badd-49f0-be2e-8fb0a4ae633d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:11.143798 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbfe0df-ecce-40ec-8dd6-5a20c4386677 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:11.210481 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2367e122-badd-49f0-be2e-8fb0a4ae633d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:11.211022 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdf5177-6638-4228-9d34-6e2ad40579c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:11.211319 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf5177-6638-4228-9d34-6e2ad40579c5 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:11.211599 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf5177-6638-4228-9d34-6e2ad40579c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:11.211867 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf5177-6638-4228-9d34-6e2ad40579c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:11.227076 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdf5177-6638-4228-9d34-6e2ad40579c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:11.228550 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdf5177-6638-4228-9d34-6e2ad40579c5 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:11.246689 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235290f4-3a3c-44b1-a64a-399af1e3c8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:11.247323 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235290f4-3a3c-44b1-a64a-399af1e3c8ce None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:11.247598 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235290f4-3a3c-44b1-a64a-399af1e3c8ce None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:11.247866 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235290f4-3a3c-44b1-a64a-399af1e3c8ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:11.251846 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-ffdf5177-6638-4228-9d34-6e2ad40579c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:11.251846 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:11.251846 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:11.251846 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:11.251846 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:11.251846 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:11.251846 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:11.251846 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:11.251846 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:11.251846 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:11.251846 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:11.251846 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:11.251846 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 05:06:11.251846 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 05:06:11.251846 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 05:06:11.252668 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 05:06:11.252668 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:11.252668 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:11.252668 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:11.252668 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:11.252668 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:11.252668 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:11.252668 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:11.252668 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:11.252668 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:11.252668 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:11.252668 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:11.252668 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:11.252668 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:11.252668 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:11.252668 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:11.252668 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:11.253510 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 05:06:11.253510 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 05:06:11.253510 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 05:06:11.253510 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 05:06:11.253510 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 05:06:11.253510 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 05:06:11.253510 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:11.253510 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:11.255050 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235290f4-3a3c-44b1-a64a-399af1e3c8ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:11.256623 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235290f4-3a3c-44b1-a64a-399af1e3c8ce None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:11.258448 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 261/536] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:06:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:11.264302 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdf5177-6638-4228-9d34-6e2ad40579c5 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:11.268650 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-235290f4-3a3c-44b1-a64a-399af1e3c8ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:11.268650 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:11.268650 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:11.268650 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:11.268650 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:11.268650 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:11.268650 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:11.268650 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:11.268650 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:11.268650 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:11.268650 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:11.268650 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:11.268650 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 05:06:11.268650 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 05:06:11.268650 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 05:06:11.270685 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 05:06:11.270685 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:11.270685 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:11.270685 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:11.270685 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:11.270685 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:11.270685 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:11.270685 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:11.270685 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:11.270685 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:11.270685 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:11.270685 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:11.270685 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:11.270685 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:11.270685 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:11.270685 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:11.270685 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:11.272618 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 05:06:11.272618 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 05:06:11.272618 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 05:06:11.272618 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 05:06:11.272618 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 05:06:11.272618 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 05:06:11.272618 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:11.272618 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:11.272618 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 276/537] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:06:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:06:11.279317 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235290f4-3a3c-44b1-a64a-399af1e3c8ce None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:11.322381 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:11.373418 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:11.417471 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdf5177-6638-4228-9d34-6e2ad40579c5 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:11.452660 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235290f4-3a3c-44b1-a64a-399af1e3c8ce None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:11.476382 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de91921c-23bb-4aa6-9c95-ecd4b4300e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:11.476996 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91921c-23bb-4aa6-9c95-ecd4b4300e36 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:11.477273 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91921c-23bb-4aa6-9c95-ecd4b4300e36 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:11.477541 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91921c-23bb-4aa6-9c95-ecd4b4300e36 None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:11.478444 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de91921c-23bb-4aa6-9c95-ecd4b4300e36 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:11.478808 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de91921c-23bb-4aa6-9c95-ecd4b4300e36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:11.480380 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de91921c-23bb-4aa6-9c95-ecd4b4300e36 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:11.500372 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39531a4c-f03c-4a52-a433-c527a4faf9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:11.501167 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39531a4c-f03c-4a52-a433-c527a4faf9ce None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:11.501576 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39531a4c-f03c-4a52-a433-c527a4faf9ce None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:11.501975 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39531a4c-f03c-4a52-a433-c527a4faf9ce None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:11.502840 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39531a4c-f03c-4a52-a433-c527a4faf9ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:11.507851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 262/538] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 05:06:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:06:11.508359 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39531a4c-f03c-4a52-a433-c527a4faf9ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:11.509980 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39531a4c-f03c-4a52-a433-c527a4faf9ce None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:11.513074 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de91921c-23bb-4aa6-9c95-ecd4b4300e36 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:11.523916 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 277/539] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 05:06:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:11.534439 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39531a4c-f03c-4a52-a433-c527a4faf9ce None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:11.635120 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39531a4c-f03c-4a52-a433-c527a4faf9ce None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:11.654901 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:11.712005 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6556f271-5ab0-44c4-b782-c748de1e671b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:11.713022 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6556f271-5ab0-44c4-b782-c748de1e671b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:11.714428 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6556f271-5ab0-44c4-b782-c748de1e671b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:11.714889 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:11.715661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6556f271-5ab0-44c4-b782-c748de1e671b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:11.717369 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6556f271-5ab0-44c4-b782-c748de1e671b None admin] RBAC: Adding query filter params (role.id=207bf98e31774cc0b21249de6d96155b, scope.project.id=e01ce1b1282e44a39d659b87f120995e, user.id=a8300075371341a6b9cd4dcf8ec7fa8f) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:11.718607 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6556f271-5ab0-44c4-b782-c748de1e671b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:11.720234 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6556f271-5ab0-44c4-b782-c748de1e671b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:11.728022 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de91921c-23bb-4aa6-9c95-ecd4b4300e36 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:11.733242 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 278/540] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:06:11 2022] GET /identity/v3/role_assignments?user.id=a8300075371341a6b9cd4dcf8ec7fa8f&scope.project.id=e01ce1b1282e44a39d659b87f120995e&role.id=207bf98e31774cc0b21249de6d96155b => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:11.781095 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8dd3da-da9d-41e9-8f85-854707aa141a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:11.782036 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8dd3da-da9d-41e9-8f85-854707aa141a None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:11.782525 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8dd3da-da9d-41e9-8f85-854707aa141a None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:11.782940 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8dd3da-da9d-41e9-8f85-854707aa141a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:11.783828 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8dd3da-da9d-41e9-8f85-854707aa141a None admin] RBAC: Adding query filter params (role.id=207bf98e31774cc0b21249de6d96155b, scope.project.id=e01ce1b1282e44a39d659b87f120995e, user.id=090e1409506542f6831e1f65d44aae74) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:11.784296 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8dd3da-da9d-41e9-8f85-854707aa141a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:11.785895 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8dd3da-da9d-41e9-8f85-854707aa141a None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:11.792591 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 263/541] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:06:11 2022] GET /identity/v3/role_assignments?user.id=090e1409506542f6831e1f65d44aae74&scope.project.id=e01ce1b1282e44a39d659b87f120995e&role.id=207bf98e31774cc0b21249de6d96155b => generated 259 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:06:12.218905 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98082b3-7c3b-48c0-9099-081c52d44df2 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:12.219436 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98082b3-7c3b-48c0-9099-081c52d44df2 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:12.219436 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98082b3-7c3b-48c0-9099-081c52d44df2 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:12.221017 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 279/542] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:12.226282 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b824325-1eb8-47e7-b305-0f73377b8972 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:12.226712 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b824325-1eb8-47e7-b305-0f73377b8972 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:12.226854 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b824325-1eb8-47e7-b305-0f73377b8972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:12.245397 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-0b824325-1eb8-47e7-b305-0f73377b8972 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:12.246613 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-0b824325-1eb8-47e7-b305-0f73377b8972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:12.331978 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 264/543] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:06:12.592918 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98082b3-7c3b-48c0-9099-081c52d44df2 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:12.735299 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98082b3-7c3b-48c0-9099-081c52d44df2 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:12.781659 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f444783c-6c44-4d33-a7c8-fe61f50fc304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:12.782314 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f444783c-6c44-4d33-a7c8-fe61f50fc304 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:12.782589 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f444783c-6c44-4d33-a7c8-fe61f50fc304 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:12.782848 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f444783c-6c44-4d33-a7c8-fe61f50fc304 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:12.802155 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f444783c-6c44-4d33-a7c8-fe61f50fc304 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:06:12.807310 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-f444783c-6c44-4d33-a7c8-fe61f50fc304 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 05:06:12.807310 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:12.807310 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:12.807310 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:12.807310 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:12.807310 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:12.807310 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:12.807310 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:12.807310 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:12.807310 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:12.807310 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:12.807310 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:12.807310 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:06:12.807310 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:06:12.807310 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:06:12.807310 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:06:12.808171 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:06:12.808171 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:06:12.808171 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:06:12.808171 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:06:12.808171 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:12.808171 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:12.808171 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:12.808171 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:12.808171 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:12.808171 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:12.808171 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:12.808171 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:12.808171 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:12.808171 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:12.808171 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:12.808171 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:12.808973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:12.808973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:12.808973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:12.808973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:12.808973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:06:12.808973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:06:12.808973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:06:12.808973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:06:12.808973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:06:12.808973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:06:12.808973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 05:06:12.808973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:12.808973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 280/544] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 05:06:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:12.814575 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b824325-1eb8-47e7-b305-0f73377b8972 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:12.857592 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055daf4-f486-47a9-8a52-0b09bf1f1952 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:12.857784 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055daf4-f486-47a9-8a52-0b09bf1f1952 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:12.858071 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055daf4-f486-47a9-8a52-0b09bf1f1952 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:12.859954 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 281/545] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 05:06:12.873753 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c37ec-2602-4a8d-a25d-99eb223f387c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:12.874053 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c37ec-2602-4a8d-a25d-99eb223f387c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:12.874333 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c37ec-2602-4a8d-a25d-99eb223f387c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:12.907282 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b54c37ec-2602-4a8d-a25d-99eb223f387c None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:12.908473 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b54c37ec-2602-4a8d-a25d-99eb223f387c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:12.946371 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:12.952141 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:12.965172 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b824325-1eb8-47e7-b305-0f73377b8972 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:12.998941 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 282/546] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 05:06:13.000482 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd87c9-1542-47e2-a611-d0603fb7135b None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:13.000994 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd87c9-1542-47e2-a611-d0603fb7135b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:13.001482 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd87c9-1542-47e2-a611-d0603fb7135b None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:13.003262 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 283/547] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:13.008386 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab55bd-c49a-4af5-8aaf-f788a6e295d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:13.008781 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb28165-96ad-41b7-b2b1-7f747dec5c65 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:13.009006 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab55bd-c49a-4af5-8aaf-f788a6e295d3 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:13.009280 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab55bd-c49a-4af5-8aaf-f788a6e295d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:13.009548 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab55bd-c49a-4af5-8aaf-f788a6e295d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:13.009872 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb28165-96ad-41b7-b2b1-7f747dec5c65 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:13.010241 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab55bd-c49a-4af5-8aaf-f788a6e295d3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:13.010541 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab55bd-c49a-4af5-8aaf-f788a6e295d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:13.010791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb28165-96ad-41b7-b2b1-7f747dec5c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:13.011909 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab55bd-c49a-4af5-8aaf-f788a6e295d3 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:13.029570 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 265/548] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Aug 31 05:06:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:13.030188 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-6cb28165-96ad-41b7-b2b1-7f747dec5c65 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:13.031578 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6cb28165-96ad-41b7-b2b1-7f747dec5c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:13.034588 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab55bd-c49a-4af5-8aaf-f788a6e295d3 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:13.107159 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 284/549] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 05:06:13.221874 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab55bd-c49a-4af5-8aaf-f788a6e295d3 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:13.272498 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aab3de-06bb-4e45-ab16-fb2bf320a6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:13.273149 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aab3de-06bb-4e45-ab16-fb2bf320a6d0 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:13.273380 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aab3de-06bb-4e45-ab16-fb2bf320a6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:13.273636 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aab3de-06bb-4e45-ab16-fb2bf320a6d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:13.291832 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aab3de-06bb-4e45-ab16-fb2bf320a6d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:13.293286 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aab3de-06bb-4e45-ab16-fb2bf320a6d0 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:13.311788 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-57aab3de-06bb-4e45-ab16-fb2bf320a6d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:13.311788 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:13.311788 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:13.311788 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:13.311788 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:13.311788 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:13.311788 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:13.311788 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:13.311788 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:13.311788 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:13.311788 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:13.311788 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:13.311788 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:13.311788 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:13.311788 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:13.311788 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:13.312699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:13.312699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:13.312699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:13.312699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:13.312699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:13.312699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:13.312699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:13.312699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:13.312699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:13.312699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:13.312699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:13.312699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:13.312699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:13.312699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:13.312699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:13.312699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:13.312699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:13.313529 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:13.313529 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:13.313529 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:13.313529 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 266/550] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:06:13.318827 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb28165-96ad-41b7-b2b1-7f747dec5c65 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:13.325846 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aab3de-06bb-4e45-ab16-fb2bf320a6d0 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:13.329089 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:13.459004 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:13.470612 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb28165-96ad-41b7-b2b1-7f747dec5c65 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:13.492827 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aab3de-06bb-4e45-ab16-fb2bf320a6d0 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:13.508996 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a29da87-55fe-4a7f-a24f-5eae73a02c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:13.509609 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29da87-55fe-4a7f-a24f-5eae73a02c6f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:13.509883 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29da87-55fe-4a7f-a24f-5eae73a02c6f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:13.510189 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29da87-55fe-4a7f-a24f-5eae73a02c6f None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:13.510955 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a29da87-55fe-4a7f-a24f-5eae73a02c6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:13.511342 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a29da87-55fe-4a7f-a24f-5eae73a02c6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:13.512816 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a29da87-55fe-4a7f-a24f-5eae73a02c6f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:13.525366 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 285/551] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:13.526296 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a29da87-55fe-4a7f-a24f-5eae73a02c6f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:13.540302 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39b017a-c8ff-4dc4-a34d-7b7342dbb20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:13.541429 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b017a-c8ff-4dc4-a34d-7b7342dbb20a None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:13.541848 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b017a-c8ff-4dc4-a34d-7b7342dbb20a None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:13.542315 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b017a-c8ff-4dc4-a34d-7b7342dbb20a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:13.557919 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39b017a-c8ff-4dc4-a34d-7b7342dbb20a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:06:13.565866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-a39b017a-c8ff-4dc4-a34d-7b7342dbb20a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 05:06:13.565866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:13.565866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:13.565866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:13.565866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:13.565866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:13.565866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:13.565866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:13.565866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:13.565866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:13.565866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:13.565866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:13.565866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:06:13.565866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:06:13.565866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:06:13.565866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:06:13.567170 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:06:13.567170 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:06:13.567170 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:06:13.567170 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:06:13.567170 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:13.567170 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:13.567170 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:13.567170 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:13.567170 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:13.567170 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:13.567170 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:13.567170 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:13.567170 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:13.567170 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:13.567170 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:13.567170 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:13.568212 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:13.568212 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:13.568212 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:13.568212 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:13.568212 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:06:13.568212 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:06:13.568212 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:06:13.568212 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:06:13.568212 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:06:13.568212 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:06:13.568212 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 05:06:13.568212 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:13.571039 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 267/552] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 05:06:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:06:13.571986 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39b017a-c8ff-4dc4-a34d-7b7342dbb20a None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:13.637334 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a29da87-55fe-4a7f-a24f-5eae73a02c6f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:13.656720 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:13.674223 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55e9935-ffeb-4bc2-b1bf-d0f98f1dea98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:13.674958 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e9935-ffeb-4bc2-b1bf-d0f98f1dea98 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:13.675300 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e9935-ffeb-4bc2-b1bf-d0f98f1dea98 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:13.675633 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e9935-ffeb-4bc2-b1bf-d0f98f1dea98 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:13.689190 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55e9935-ffeb-4bc2-b1bf-d0f98f1dea98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:06:13.694976 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-d55e9935-ffeb-4bc2-b1bf-d0f98f1dea98 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 05:06:13.694976 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:13.694976 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:13.694976 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:13.694976 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:13.694976 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:13.694976 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:13.694976 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:13.694976 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:13.694976 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:13.694976 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:13.694976 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:13.694976 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:06:13.694976 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:06:13.694976 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:06:13.694976 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:06:13.696115 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:06:13.696115 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:06:13.696115 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:06:13.696115 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:06:13.696115 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:13.696115 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:13.696115 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:13.696115 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:13.696115 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:13.696115 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:13.696115 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:13.696115 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:13.696115 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:13.696115 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:13.696115 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:13.696115 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:13.697332 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:13.697332 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:13.697332 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:13.697332 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:13.697332 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:06:13.697332 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:06:13.697332 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:06:13.697332 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:06:13.697332 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:06:13.697332 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:06:13.697332 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 05:06:13.697332 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:13.701301 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 286/553] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 05:06:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:13.702306 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55e9935-ffeb-4bc2-b1bf-d0f98f1dea98 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:13.705726 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39b017a-c8ff-4dc4-a34d-7b7342dbb20a None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:13.716039 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:13.752266 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce42cc02-cf78-4180-9b08-1492d6737552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:13.753163 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42cc02-cf78-4180-9b08-1492d6737552 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:13.753596 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42cc02-cf78-4180-9b08-1492d6737552 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:13.754081 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42cc02-cf78-4180-9b08-1492d6737552 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:13.762896 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce42cc02-cf78-4180-9b08-1492d6737552 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:13.764701 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce42cc02-cf78-4180-9b08-1492d6737552 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:13.778416 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-ce42cc02-cf78-4180-9b08-1492d6737552 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 05:06:13.778416 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:13.778416 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:13.778416 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:13.778416 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:13.778416 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:13.778416 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:13.778416 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:13.778416 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:13.778416 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:13.778416 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:13.778416 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:13.778416 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 05:06:13.778416 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 05:06:13.778416 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 05:06:13.778416 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 05:06:13.780103 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:13.780103 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:13.780103 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 05:06:13.780103 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:13.780103 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 05:06:13.780103 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:13.780103 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:13.780103 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:13.780103 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:13.780103 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:13.780103 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:13.780103 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:13.780103 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:13.780103 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:13.780103 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:13.780103 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:13.780103 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:13.780103 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:13.781720 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:13.781720 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:13.781720 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 05:06:13.781720 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 05:06:13.781720 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 05:06:13.781720 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 05:06:13.781720 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 05:06:13.781720 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 05:06:13.781720 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 05:06:13.781720 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:13.783673 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 268/554] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 05:06:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:13.784656 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce42cc02-cf78-4180-9b08-1492d6737552 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:13.820279 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55e9935-ffeb-4bc2-b1bf-d0f98f1dea98 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:13.856319 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa971702-6b2b-4929-8234-8e91bc669088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:13.857225 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa971702-6b2b-4929-8234-8e91bc669088 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:13.857336 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa971702-6b2b-4929-8234-8e91bc669088 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:13.857678 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa971702-6b2b-4929-8234-8e91bc669088 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:13.858404 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa971702-6b2b-4929-8234-8e91bc669088 None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:13.858663 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa971702-6b2b-4929-8234-8e91bc669088 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:13.860041 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa971702-6b2b-4929-8234-8e91bc669088 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:13.873766 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 287/555] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Aug 31 05:06:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:13.874686 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa971702-6b2b-4929-8234-8e91bc669088 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:13.900394 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce42cc02-cf78-4180-9b08-1492d6737552 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:13.926764 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:13.939830 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cbfa18-a2a6-4dce-9007-ef46fcc315a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:13.940742 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cbfa18-a2a6-4dce-9007-ef46fcc315a2 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:13.941271 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cbfa18-a2a6-4dce-9007-ef46fcc315a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:13.941830 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cbfa18-a2a6-4dce-9007-ef46fcc315a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:13.942738 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cbfa18-a2a6-4dce-9007-ef46fcc315a2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:13.943263 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cbfa18-a2a6-4dce-9007-ef46fcc315a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:13.944872 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cbfa18-a2a6-4dce-9007-ef46fcc315a2 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:13.949846 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:13.956461 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 269/556] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Aug 31 05:06:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:13.957519 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cbfa18-a2a6-4dce-9007-ef46fcc315a2 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:13.986232 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa971702-6b2b-4929-8234-8e91bc669088 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:14.033330 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2404682e-9689-484f-ba1f-7f353ea5af3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:14.033859 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2404682e-9689-484f-ba1f-7f353ea5af3d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:14.034167 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2404682e-9689-484f-ba1f-7f353ea5af3d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:14.034434 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2404682e-9689-484f-ba1f-7f353ea5af3d None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:14.035425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2404682e-9689-484f-ba1f-7f353ea5af3d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:14.035729 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2404682e-9689-484f-ba1f-7f353ea5af3d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:14.037390 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2404682e-9689-484f-ba1f-7f353ea5af3d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:14.069906 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cbfa18-a2a6-4dce-9007-ef46fcc315a2 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:14.082047 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 288/557] 10.222.0.36 () {56 vars in 1135 bytes} [Wed Aug 31 05:06:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:14.082895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2404682e-9689-484f-ba1f-7f353ea5af3d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:14.109724 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f073909-a564-4033-849e-b7308c37b35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:14.110615 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f073909-a564-4033-849e-b7308c37b35e None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:14.111109 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f073909-a564-4033-849e-b7308c37b35e None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:14.111630 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f073909-a564-4033-849e-b7308c37b35e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:14.125315 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f073909-a564-4033-849e-b7308c37b35e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:14.126808 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f073909-a564-4033-849e-b7308c37b35e None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:14.144757 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-9f073909-a564-4033-849e-b7308c37b35e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:14.144757 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:14.144757 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:14.144757 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:14.144757 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:14.144757 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:14.144757 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:14.144757 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:14.144757 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:14.144757 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:14.144757 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:14.144757 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:14.144757 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:14.144757 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:14.144757 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:14.144757 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:14.145776 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:14.145776 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:14.145776 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:14.145776 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:14.145776 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:14.145776 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:14.145776 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:14.145776 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:14.145776 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:14.145776 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:14.145776 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:14.145776 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:14.145776 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:14.145776 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:14.145776 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:14.145776 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:14.145776 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:14.147351 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:14.147351 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:14.147351 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:14.147351 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 270/558] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:14.147351 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5eae3-5d27-4114-9f52-77bbcaba8bff None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:14.147351 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5eae3-5d27-4114-9f52-77bbcaba8bff None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:14.147692 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5eae3-5d27-4114-9f52-77bbcaba8bff None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:14.153391 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 271/559] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:14.153391 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e5eae3-5d27-4114-9f52-77bbcaba8bff None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:14.191565 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:14.194155 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2404682e-9689-484f-ba1f-7f353ea5af3d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:14.224905 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:14.231068 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90f57d-f1c0-431b-9c6d-5ddd22c08cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:14.231703 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90f57d-f1c0-431b-9c6d-5ddd22c08cef None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:14.231973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90f57d-f1c0-431b-9c6d-5ddd22c08cef None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:14.232237 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90f57d-f1c0-431b-9c6d-5ddd22c08cef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:14.242121 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f90f57d-f1c0-431b-9c6d-5ddd22c08cef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:14.243883 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f90f57d-f1c0-431b-9c6d-5ddd22c08cef None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:14.259361 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-7f90f57d-f1c0-431b-9c6d-5ddd22c08cef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:14.259361 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:14.259361 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:14.259361 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:14.259361 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:14.259361 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:14.259361 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:14.259361 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:14.259361 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:14.259361 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:14.259361 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:14.259361 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:14.259361 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:14.259361 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:14.259361 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:14.259361 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:14.260374 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:14.260374 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:14.260374 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:14.260374 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:14.260374 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:14.260374 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:14.260374 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:14.260374 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:14.260374 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:14.260374 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:14.260374 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:14.260374 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:14.260374 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:14.260374 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:14.260374 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:14.260374 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:14.260374 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:14.261276 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:14.261276 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:14.261276 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:14.261276 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 289/560] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:14.261276 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90f57d-f1c0-431b-9c6d-5ddd22c08cef None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:14.266813 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e5eae3-5d27-4114-9f52-77bbcaba8bff None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:14.301032 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b25d8f-cfa6-43b9-a7d0-80c6f4d55f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:14.301635 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b25d8f-cfa6-43b9-a7d0-80c6f4d55f5c None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:14.301819 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b25d8f-cfa6-43b9-a7d0-80c6f4d55f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:14.302129 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b25d8f-cfa6-43b9-a7d0-80c6f4d55f5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:14.358835 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90f57d-f1c0-431b-9c6d-5ddd22c08cef None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:14.370032 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b25d8f-cfa6-43b9-a7d0-80c6f4d55f5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:14.371520 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b25d8f-cfa6-43b9-a7d0-80c6f4d55f5c None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:14.391864 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a1862d-f8af-4234-9827-b836582f687e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:14.392470 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1862d-f8af-4234-9827-b836582f687e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:14.392745 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1862d-f8af-4234-9827-b836582f687e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:14.393012 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1862d-f8af-4234-9827-b836582f687e None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:14.393654 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a1862d-f8af-4234-9827-b836582f687e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:14.393956 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a1862d-f8af-4234-9827-b836582f687e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:14.395365 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a1862d-f8af-4234-9827-b836582f687e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:14.404243 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:14.407641 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 290/561] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:14.408806 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177f346-e246-4926-915c-95def5d9a31f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:14.409089 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177f346-e246-4926-915c-95def5d9a31f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:14.409360 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177f346-e246-4926-915c-95def5d9a31f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:14.426876 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-4177f346-e246-4926-915c-95def5d9a31f None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:14.428116 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4177f346-e246-4926-915c-95def5d9a31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:14.460989 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:14.468963 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 272/562] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:06:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 05:06:14.470243 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4427520-a61d-4d4a-8873-3f0e050354a5 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:14.470527 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4427520-a61d-4d4a-8873-3f0e050354a5 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:14.470797 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4427520-a61d-4d4a-8873-3f0e050354a5 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:14.472196 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 273/563] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:14.472729 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4427520-a61d-4d4a-8873-3f0e050354a5 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:14.498281 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 291/564] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:06:14.499099 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4177f346-e246-4926-915c-95def5d9a31f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:14.577435 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4427520-a61d-4d4a-8873-3f0e050354a5 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:14.598401 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4177f346-e246-4926-915c-95def5d9a31f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:14.611887 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c32d7a3-ca67-464e-bd97-51cbba2523e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:14.612495 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32d7a3-ca67-464e-bd97-51cbba2523e6 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:14.612767 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32d7a3-ca67-464e-bd97-51cbba2523e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:14.613031 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32d7a3-ca67-464e-bd97-51cbba2523e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:14.613665 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c32d7a3-ca67-464e-bd97-51cbba2523e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:14.613959 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c32d7a3-ca67-464e-bd97-51cbba2523e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:14.615374 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c32d7a3-ca67-464e-bd97-51cbba2523e6 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:14.618122 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:14.629538 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 274/565] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:14.630237 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c32d7a3-ca67-464e-bd97-51cbba2523e6 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:14.636962 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1555de01-ab62-49ae-a669-c210fb13198d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:14.637633 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555de01-ab62-49ae-a669-c210fb13198d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:14.640741 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555de01-ab62-49ae-a669-c210fb13198d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:14.640741 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555de01-ab62-49ae-a669-c210fb13198d None admin] PATH_INFO: `/v3/users/glance` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:14.649837 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1555de01-ab62-49ae-a669-c210fb13198d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:14.653102 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1555de01-ab62-49ae-a669-c210fb13198d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:14.668283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-1555de01-ab62-49ae-a669-c210fb13198d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 05:06:14.668283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:14.668283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:14.668283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:14.668283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:14.668283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:14.668283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:14.668283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:14.668283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:14.668283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:14.668283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:14.668283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:14.668283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 05:06:14.668283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 05:06:14.668283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 05:06:14.668283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 05:06:14.669243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:14.669243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:14.669243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 05:06:14.669243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:14.669243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 05:06:14.669243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:14.669243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:14.669243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:14.669243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:14.669243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:14.669243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:14.669243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:14.669243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:14.669243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:14.669243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:14.669243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:14.669243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:14.669243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:14.670281 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:14.670281 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:14.670281 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 05:06:14.670281 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 05:06:14.670281 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 05:06:14.670281 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 05:06:14.670281 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 05:06:14.670281 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 05:06:14.670281 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 05:06:14.670281 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:14.674151 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 292/566] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:06:14 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:14.676351 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d5070-23fa-4c1b-96c0-6ba1ae76205e None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:14.676634 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d5070-23fa-4c1b-96c0-6ba1ae76205e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:14.677077 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d5070-23fa-4c1b-96c0-6ba1ae76205e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:14.698294 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-5b2d5070-23fa-4c1b-96c0-6ba1ae76205e None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:14.699460 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5b2d5070-23fa-4c1b-96c0-6ba1ae76205e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:14.727772 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:14.730323 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c32d7a3-ca67-464e-bd97-51cbba2523e6 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:14.764363 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f011ea8-5b05-4d6a-88d6-45489d939e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:14.764978 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f011ea8-5b05-4d6a-88d6-45489d939e20 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:14.765269 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f011ea8-5b05-4d6a-88d6-45489d939e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:14.765532 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f011ea8-5b05-4d6a-88d6-45489d939e20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:14.772763 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 293/567] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:06:14.773596 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d5070-23fa-4c1b-96c0-6ba1ae76205e None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:14.775552 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f011ea8-5b05-4d6a-88d6-45489d939e20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:14.777285 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f011ea8-5b05-4d6a-88d6-45489d939e20 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:14.792113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-0f011ea8-5b05-4d6a-88d6-45489d939e20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:14.792113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:14.792113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:14.792113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:14.792113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:14.792113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:14.792113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:14.792113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:14.792113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:14.792113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:14.792113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:14.792113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:14.792113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:14.792113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:14.792113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:14.792113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:14.793050 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:14.793050 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:14.793050 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:14.793050 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:14.793050 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:14.793050 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:14.793050 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:14.793050 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:14.793050 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:14.793050 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:14.793050 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:14.793050 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:14.793050 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:14.793050 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:14.793050 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:14.793050 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:14.793050 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:14.793926 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:14.793926 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:14.793926 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:14.793926 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 275/568] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:14.794190 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f011ea8-5b05-4d6a-88d6-45489d939e20 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:14.851647 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:14.880623 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d5070-23fa-4c1b-96c0-6ba1ae76205e None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:14.911752 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f011ea8-5b05-4d6a-88d6-45489d939e20 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:14.916160 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b9533-7816-47dd-8351-6a6bc8b71674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:14.916790 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b9533-7816-47dd-8351-6a6bc8b71674 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:14.917063 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b9533-7816-47dd-8351-6a6bc8b71674 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:14.917327 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b9533-7816-47dd-8351-6a6bc8b71674 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:14.924582 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066b9533-7816-47dd-8351-6a6bc8b71674 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:14.926038 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066b9533-7816-47dd-8351-6a6bc8b71674 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:14.937478 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-066b9533-7816-47dd-8351-6a6bc8b71674 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 05:06:14.937478 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:14.937478 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:14.937478 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:14.937478 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:14.937478 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:14.937478 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:14.937478 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:14.937478 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:14.937478 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:14.937478 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:14.937478 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:14.937478 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 05:06:14.937478 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 05:06:14.937478 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 05:06:14.937478 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 05:06:14.938428 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:14.938428 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:14.938428 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 05:06:14.938428 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:14.938428 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 05:06:14.938428 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:14.938428 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:14.938428 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:14.938428 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:14.938428 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:14.938428 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:14.938428 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:14.938428 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:14.938428 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:14.938428 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:14.938428 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:14.938428 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:14.938428 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:14.939386 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:14.939386 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:14.939386 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 05:06:14.939386 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 05:06:14.939386 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 05:06:14.939386 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 05:06:14.939386 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 05:06:14.939386 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 05:06:14.939386 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 05:06:14.939386 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:14.939386 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 294/569] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Aug 31 05:06:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:14.939386 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b9533-7816-47dd-8351-6a6bc8b71674 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:14.953542 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94759a92-e747-4f71-843a-65bfc617cd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:14.954250 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94759a92-e747-4f71-843a-65bfc617cd75 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:14.954682 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94759a92-e747-4f71-843a-65bfc617cd75 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:14.955008 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94759a92-e747-4f71-843a-65bfc617cd75 None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:14.955741 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94759a92-e747-4f71-843a-65bfc617cd75 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:14.956037 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94759a92-e747-4f71-843a-65bfc617cd75 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:14.957586 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94759a92-e747-4f71-843a-65bfc617cd75 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:14.967039 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:14.981982 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 276/570] 10.222.0.36 () {56 vars in 1137 bytes} [Wed Aug 31 05:06:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:14.981982 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94759a92-e747-4f71-843a-65bfc617cd75 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:15.052117 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b9533-7816-47dd-8351-6a6bc8b71674 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:15.078326 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94759a92-e747-4f71-843a-65bfc617cd75 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:15.084087 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce0dd55-686d-478c-b224-a5c1532e7f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:15.084705 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce0dd55-686d-478c-b224-a5c1532e7f26 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:15.084976 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce0dd55-686d-478c-b224-a5c1532e7f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:15.085240 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce0dd55-686d-478c-b224-a5c1532e7f26 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:15.085551 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:15.095341 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce0dd55-686d-478c-b224-a5c1532e7f26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:06:15.100558 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-9ce0dd55-686d-478c-b224-a5c1532e7f26 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 05:06:15.100558 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:15.100558 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:15.100558 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:15.100558 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:15.100558 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:15.100558 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:15.100558 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:15.100558 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:15.100558 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:15.100558 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:15.100558 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:15.100558 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:06:15.100558 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:06:15.100558 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:06:15.100558 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:06:15.101489 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:06:15.101489 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:06:15.101489 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:06:15.101489 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:06:15.101489 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:15.101489 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:15.101489 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:15.101489 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:15.101489 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:15.101489 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:15.101489 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:15.101489 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:15.101489 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:15.101489 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:15.101489 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:15.101489 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:15.102683 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:15.102683 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:15.102683 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:15.102683 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:15.102683 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:06:15.102683 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:06:15.102683 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:06:15.102683 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:06:15.102683 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:06:15.102683 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:06:15.102683 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 05:06:15.102683 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:15.102683 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 295/571] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 05:06:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:15.102683 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce0dd55-686d-478c-b224-a5c1532e7f26 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:15.117074 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0a388-22a5-4a05-8474-91c6aa47dc5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:15.117678 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0a388-22a5-4a05-8474-91c6aa47dc5c None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:15.117937 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0a388-22a5-4a05-8474-91c6aa47dc5c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:15.118548 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0a388-22a5-4a05-8474-91c6aa47dc5c None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:15.119188 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd0a388-22a5-4a05-8474-91c6aa47dc5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:15.119488 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd0a388-22a5-4a05-8474-91c6aa47dc5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:15.120870 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd0a388-22a5-4a05-8474-91c6aa47dc5c None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:15.132033 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 277/572] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:15.132813 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0a388-22a5-4a05-8474-91c6aa47dc5c None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:15.183356 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:15.203345 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce0dd55-686d-478c-b224-a5c1532e7f26 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:15.229774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0a388-22a5-4a05-8474-91c6aa47dc5c None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:15.235977 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae05ba1-6973-4f06-9bc1-ca464772c111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:15.236573 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae05ba1-6973-4f06-9bc1-ca464772c111 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:15.236850 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae05ba1-6973-4f06-9bc1-ca464772c111 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:15.237149 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae05ba1-6973-4f06-9bc1-ca464772c111 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:15.237879 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae05ba1-6973-4f06-9bc1-ca464772c111 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:15.238213 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae05ba1-6973-4f06-9bc1-ca464772c111 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:15.239654 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae05ba1-6973-4f06-9bc1-ca464772c111 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:15.257057 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 296/573] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Aug 31 05:06:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:15.257839 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae05ba1-6973-4f06-9bc1-ca464772c111 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:15.263068 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438517c4-d4ba-4aea-8cff-d371c2de3724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:15.263691 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438517c4-d4ba-4aea-8cff-d371c2de3724 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:15.263987 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438517c4-d4ba-4aea-8cff-d371c2de3724 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:15.264251 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438517c4-d4ba-4aea-8cff-d371c2de3724 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:15.274767 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438517c4-d4ba-4aea-8cff-d371c2de3724 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:15.276229 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438517c4-d4ba-4aea-8cff-d371c2de3724 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:15.290868 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-438517c4-d4ba-4aea-8cff-d371c2de3724 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:15.290868 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:15.290868 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:15.290868 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:15.290868 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:15.290868 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:15.290868 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:15.290868 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:15.290868 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:15.290868 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:15.290868 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:15.290868 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:15.290868 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:15.290868 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:15.290868 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:15.290868 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:15.291819 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:15.291819 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:15.291819 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:15.291819 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:15.291819 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:15.291819 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:15.291819 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:15.291819 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:15.291819 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:15.291819 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:15.291819 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:15.291819 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:15.291819 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:15.291819 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:15.291819 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:15.291819 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:15.291819 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:15.293051 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:15.293051 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:15.293051 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:15.293051 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 278/574] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:15.293051 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438517c4-d4ba-4aea-8cff-d371c2de3724 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:15.312723 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:15.358354 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae05ba1-6973-4f06-9bc1-ca464772c111 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:15.390547 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438517c4-d4ba-4aea-8cff-d371c2de3724 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:15.392184 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c050b541-ac26-480e-a2e9-eb3269f9fae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:15.392809 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050b541-ac26-480e-a2e9-eb3269f9fae1 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:15.393081 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050b541-ac26-480e-a2e9-eb3269f9fae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:15.393346 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050b541-ac26-480e-a2e9-eb3269f9fae1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:15.400875 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:15.404073 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c050b541-ac26-480e-a2e9-eb3269f9fae1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:06:15.409646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-c050b541-ac26-480e-a2e9-eb3269f9fae1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 05:06:15.409646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:15.409646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:15.409646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:15.409646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:15.409646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:15.409646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:15.409646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:15.409646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:15.409646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:15.409646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:15.409646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:15.409646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:06:15.409646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:06:15.409646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:06:15.409646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:06:15.410661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:06:15.410661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:06:15.410661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:06:15.410661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:06:15.410661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:15.410661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:15.410661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:15.410661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:15.410661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:15.410661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:15.410661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:15.410661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:15.410661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:15.410661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:15.410661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:15.410661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:15.411714 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:15.411714 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:15.411714 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:15.411714 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:15.411714 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:06:15.411714 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:06:15.411714 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:06:15.411714 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:06:15.411714 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:06:15.411714 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:06:15.411714 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 05:06:15.411714 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:15.411714 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 297/575] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 05:06:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:15.411714 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c050b541-ac26-480e-a2e9-eb3269f9fae1 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:15.427709 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc391ee1-92f3-4143-8f72-90b72bf57f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:15.428430 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc391ee1-92f3-4143-8f72-90b72bf57f7d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:15.428596 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc391ee1-92f3-4143-8f72-90b72bf57f7d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:15.428865 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc391ee1-92f3-4143-8f72-90b72bf57f7d None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:15.429535 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc391ee1-92f3-4143-8f72-90b72bf57f7d None admin] RBAC: Adding query filter params (name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:15.429892 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc391ee1-92f3-4143-8f72-90b72bf57f7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:15.431369 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc391ee1-92f3-4143-8f72-90b72bf57f7d None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:15.439876 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 279/576] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Aug 31 05:06:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:15.440972 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc391ee1-92f3-4143-8f72-90b72bf57f7d None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:15.516062 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c050b541-ac26-480e-a2e9-eb3269f9fae1 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:15.521176 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:15.540813 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc391ee1-92f3-4143-8f72-90b72bf57f7d None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:15.552723 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d01a7-d274-410d-815a-d2eb50f80164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:15.553354 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d01a7-d274-410d-815a-d2eb50f80164 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:15.553635 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d01a7-d274-410d-815a-d2eb50f80164 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:15.553903 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d01a7-d274-410d-815a-d2eb50f80164 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:15.560583 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725d01a7-d274-410d-815a-d2eb50f80164 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:15.562058 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725d01a7-d274-410d-815a-d2eb50f80164 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:15.573013 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2cfad2-a2a1-4685-8cd2-407532b36b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:15.573144 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-725d01a7-d274-410d-815a-d2eb50f80164 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:15.573144 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:15.573144 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:15.573144 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:15.573144 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:15.573144 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:15.573144 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:15.573144 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:15.573144 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:15.573144 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:15.573144 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:15.573144 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:15.573144 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 05:06:15.573144 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 05:06:15.573144 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 05:06:15.574202 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 05:06:15.574202 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:15.574202 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:15.574202 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:15.574202 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:15.574202 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:15.574202 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:15.574202 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:15.574202 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:15.574202 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:15.574202 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:15.574202 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:15.574202 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:15.574202 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:15.574202 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:15.574202 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:15.574202 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:15.575174 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 05:06:15.575174 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 05:06:15.575174 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 05:06:15.575174 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 05:06:15.575174 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 05:06:15.575174 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 05:06:15.575174 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:15.575174 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:15.575601 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2cfad2-a2a1-4685-8cd2-407532b36b0f None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:15.575601 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2cfad2-a2a1-4685-8cd2-407532b36b0f None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:15.575601 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2cfad2-a2a1-4685-8cd2-407532b36b0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:15.575789 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 298/577] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:06:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:15.575789 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d01a7-d274-410d-815a-d2eb50f80164 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:15.586648 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2cfad2-a2a1-4685-8cd2-407532b36b0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:15.588139 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2cfad2-a2a1-4685-8cd2-407532b36b0f None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:15.601587 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-6d2cfad2-a2a1-4685-8cd2-407532b36b0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:15.601587 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:15.601587 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:15.601587 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:15.601587 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:15.601587 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:15.601587 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:15.601587 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:15.601587 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:15.601587 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:15.601587 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:15.601587 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:15.601587 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:15.601587 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:15.601587 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:15.601587 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:15.602777 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:15.602777 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:15.602777 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:15.602777 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:15.602777 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:15.602777 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:15.602777 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:15.602777 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:15.602777 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:15.602777 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:15.602777 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:15.602777 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:15.602777 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:15.602777 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:15.602777 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:15.602777 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:15.602777 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:15.603833 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:15.603833 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:15.603833 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:15.603833 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 280/578] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:15.603833 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2cfad2-a2a1-4685-8cd2-407532b36b0f None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:15.642689 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:15.683805 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d01a7-d274-410d-815a-d2eb50f80164 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:15.703683 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2cfad2-a2a1-4685-8cd2-407532b36b0f None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:15.717239 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3836577-ca55-483a-a80c-ada8f0869e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:15.717838 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3836577-ca55-483a-a80c-ada8f0869e20 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:15.718143 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3836577-ca55-483a-a80c-ada8f0869e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:15.718405 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3836577-ca55-483a-a80c-ada8f0869e20 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:15.719038 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3836577-ca55-483a-a80c-ada8f0869e20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:15.719335 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3836577-ca55-483a-a80c-ada8f0869e20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:15.720704 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3836577-ca55-483a-a80c-ada8f0869e20 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:15.731821 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:15.732041 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 299/579] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:15.733048 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3836577-ca55-483a-a80c-ada8f0869e20 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:15.742115 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f8a680-2ab8-458a-ba8d-a788233562c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:15.742720 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8a680-2ab8-458a-ba8d-a788233562c7 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:15.742991 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8a680-2ab8-458a-ba8d-a788233562c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:15.743563 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8a680-2ab8-458a-ba8d-a788233562c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:15.744243 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f8a680-2ab8-458a-ba8d-a788233562c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:15.744536 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f8a680-2ab8-458a-ba8d-a788233562c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:15.745886 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f8a680-2ab8-458a-ba8d-a788233562c7 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:15.754119 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 281/580] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Aug 31 05:06:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:15.754874 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f8a680-2ab8-458a-ba8d-a788233562c7 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:15.837124 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3836577-ca55-483a-a80c-ada8f0869e20 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:15.853794 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f8a680-2ab8-458a-ba8d-a788233562c7 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:15.856845 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:15.871393 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefbba46-ff7d-4e36-8250-f33bd36b6d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:15.872021 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefbba46-ff7d-4e36-8250-f33bd36b6d82 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:15.872296 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefbba46-ff7d-4e36-8250-f33bd36b6d82 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:15.872566 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefbba46-ff7d-4e36-8250-f33bd36b6d82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:15.882908 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefbba46-ff7d-4e36-8250-f33bd36b6d82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:15.884372 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefbba46-ff7d-4e36-8250-f33bd36b6d82 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:15.888516 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f70637-30ba-45b6-abbc-3b86834478d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:15.889148 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f70637-30ba-45b6-abbc-3b86834478d7 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:15.889422 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f70637-30ba-45b6-abbc-3b86834478d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:15.889687 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f70637-30ba-45b6-abbc-3b86834478d7 None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:15.890407 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f70637-30ba-45b6-abbc-3b86834478d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:15.890710 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f70637-30ba-45b6-abbc-3b86834478d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:15.892117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f70637-30ba-45b6-abbc-3b86834478d7 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:15.898268 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-eefbba46-ff7d-4e36-8250-f33bd36b6d82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:15.898268 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:15.898268 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:15.898268 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:15.898268 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:15.898268 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:15.898268 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:15.898268 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:15.898268 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:15.898268 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:15.898268 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:15.898268 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:15.898268 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:15.898268 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:15.898268 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:15.898268 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:15.899407 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:15.899407 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:15.899407 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:15.899407 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:15.899407 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:15.899407 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:15.899407 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:15.899407 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:15.899407 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:15.899407 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:15.899407 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:15.899407 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:15.899407 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:15.899407 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:15.899407 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:15.899407 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:15.899407 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:15.900378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:15.900378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:15.900378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:15.900378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 300/581] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:15.900378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefbba46-ff7d-4e36-8250-f33bd36b6d82 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:15.906971 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 282/582] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 05:06:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:15.907791 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f70637-30ba-45b6-abbc-3b86834478d7 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:15.961585 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:15.998224 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefbba46-ff7d-4e36-8250-f33bd36b6d82 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:16.006965 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f70637-30ba-45b6-abbc-3b86834478d7 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:16.031832 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afbf3f8-7f26-4170-bfb9-720a3ee7f8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:16.032414 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbf3f8-7f26-4170-bfb9-720a3ee7f8e0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:16.032690 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbf3f8-7f26-4170-bfb9-720a3ee7f8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:16.032955 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbf3f8-7f26-4170-bfb9-720a3ee7f8e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:16.033588 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afbf3f8-7f26-4170-bfb9-720a3ee7f8e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:16.033886 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afbf3f8-7f26-4170-bfb9-720a3ee7f8e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:16.035288 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afbf3f8-7f26-4170-bfb9-720a3ee7f8e0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:16.039536 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c8c36-6e1f-4c3a-8856-882b40543952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:16.040150 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c8c36-6e1f-4c3a-8856-882b40543952 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:16.040421 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c8c36-6e1f-4c3a-8856-882b40543952 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:16.040684 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c8c36-6e1f-4c3a-8856-882b40543952 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:16.046627 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 301/583] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:16.047378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afbf3f8-7f26-4170-bfb9-720a3ee7f8e0 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:16.047911 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c8c36-6e1f-4c3a-8856-882b40543952 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:16.049379 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c8c36-6e1f-4c3a-8856-882b40543952 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:16.061509 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-5c9c8c36-6e1f-4c3a-8856-882b40543952 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:16.061509 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:16.061509 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:16.061509 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:16.061509 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:16.061509 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:16.061509 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:16.061509 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:16.061509 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:16.061509 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:16.061509 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:16.061509 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:16.061509 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 05:06:16.061509 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 05:06:16.061509 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 05:06:16.062650 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 05:06:16.062650 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:16.062650 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:16.062650 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:16.062650 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:16.062650 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:16.062650 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:16.062650 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:16.062650 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:16.062650 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:16.062650 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:16.062650 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:16.062650 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:16.062650 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:16.062650 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:16.062650 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:16.062650 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:16.063678 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 05:06:16.063678 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 05:06:16.063678 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 05:06:16.063678 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 05:06:16.063678 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 05:06:16.063678 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 05:06:16.063678 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:16.063678 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:16.063678 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 283/584] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:06:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:16.063678 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c8c36-6e1f-4c3a-8856-882b40543952 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:16.080354 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:16.145979 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afbf3f8-7f26-4170-bfb9-720a3ee7f8e0 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:16.161493 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c8c36-6e1f-4c3a-8856-882b40543952 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:16.174631 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:16.180340 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d44e17-d1cc-47a3-b4fe-87fb6ab510f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:16.180971 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d44e17-d1cc-47a3-b4fe-87fb6ab510f8 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:16.181264 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d44e17-d1cc-47a3-b4fe-87fb6ab510f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:16.181524 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d44e17-d1cc-47a3-b4fe-87fb6ab510f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:16.191302 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d44e17-d1cc-47a3-b4fe-87fb6ab510f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:16.192741 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d44e17-d1cc-47a3-b4fe-87fb6ab510f8 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:16.194410 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2aadd0-c21f-4743-a811-2564491b6c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:16.195003 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2aadd0-c21f-4743-a811-2564491b6c26 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:16.195277 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2aadd0-c21f-4743-a811-2564491b6c26 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:16.195539 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2aadd0-c21f-4743-a811-2564491b6c26 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:16.196176 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2aadd0-c21f-4743-a811-2564491b6c26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:16.196470 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2aadd0-c21f-4743-a811-2564491b6c26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:16.197812 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2aadd0-c21f-4743-a811-2564491b6c26 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:16.207032 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-e4d44e17-d1cc-47a3-b4fe-87fb6ab510f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:16.207032 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:16.207032 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:16.207032 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:16.207032 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:16.207032 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:16.207032 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:16.207032 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:16.207032 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:16.207032 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:16.207032 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:16.207032 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:16.207032 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:16.207032 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:16.207032 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:16.207032 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:16.207932 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:16.207932 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:16.207932 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:16.207932 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:16.207932 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:16.207932 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:16.207932 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:16.207932 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:16.207932 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:16.207932 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:16.207932 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:16.207932 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:16.207932 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:16.207932 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:16.207932 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:16.207932 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:16.207932 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:16.208802 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:16.208802 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:16.208802 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:16.208802 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 302/585] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:16.209034 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d44e17-d1cc-47a3-b4fe-87fb6ab510f8 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:16.210171 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 284/586] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:16.210951 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2aadd0-c21f-4743-a811-2564491b6c26 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:16.293936 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:16.311217 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d44e17-d1cc-47a3-b4fe-87fb6ab510f8 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:16.311953 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2aadd0-c21f-4743-a811-2564491b6c26 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:16.343231 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7e864-f45d-46c5-85c0-769eb04da621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:16.343833 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7e864-f45d-46c5-85c0-769eb04da621 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:16.344112 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7e864-f45d-46c5-85c0-769eb04da621 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:16.344382 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7e864-f45d-46c5-85c0-769eb04da621 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:16.345135 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f7e864-f45d-46c5-85c0-769eb04da621 None admin] RBAC: Adding query filter params (role.id=207bf98e31774cc0b21249de6d96155b, scope.project.id=e01ce1b1282e44a39d659b87f120995e, user.id=d9b82080a73d4c4a8ec08b0e67d6d11c) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:16.345414 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f59792-f5f8-4799-b8af-34e47d690d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:16.345857 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f7e864-f45d-46c5-85c0-769eb04da621 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:16.346064 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f59792-f5f8-4799-b8af-34e47d690d32 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:16.346342 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f59792-f5f8-4799-b8af-34e47d690d32 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:16.346609 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f59792-f5f8-4799-b8af-34e47d690d32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:16.346883 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f7e864-f45d-46c5-85c0-769eb04da621 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:16.352185 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 303/587] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:06:16 2022] GET /identity/v3/role_assignments?user.id=d9b82080a73d4c4a8ec08b0e67d6d11c&scope.project.id=e01ce1b1282e44a39d659b87f120995e&role.id=207bf98e31774cc0b21249de6d96155b => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:16.352959 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7e864-f45d-46c5-85c0-769eb04da621 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:16.353647 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f59792-f5f8-4799-b8af-34e47d690d32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:16.355173 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f59792-f5f8-4799-b8af-34e47d690d32 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:16.369941 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-78f59792-f5f8-4799-b8af-34e47d690d32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:16.369941 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:16.369941 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:16.369941 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:16.369941 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:16.369941 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:16.369941 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:16.369941 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:16.369941 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:16.369941 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:16.369941 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:16.369941 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:16.369941 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 05:06:16.369941 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 05:06:16.369941 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 05:06:16.371055 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 05:06:16.371055 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:16.371055 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:16.371055 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:16.371055 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:16.371055 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:16.371055 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:16.371055 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:16.371055 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:16.371055 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:16.371055 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:16.371055 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:16.371055 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:16.371055 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:16.371055 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:16.371055 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:16.371055 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:16.371965 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 05:06:16.371965 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 05:06:16.371965 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 05:06:16.371965 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 05:06:16.371965 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 05:06:16.371965 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 05:06:16.371965 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:16.371965 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:16.371965 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 285/588] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:06:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:16.372483 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f59792-f5f8-4799-b8af-34e47d690d32 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:16.414255 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:16.471984 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7e864-f45d-46c5-85c0-769eb04da621 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:16.490907 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f59792-f5f8-4799-b8af-34e47d690d32 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:16.508609 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25089a1c-3c2f-4fe5-817a-1e5bc7199fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:16.509227 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25089a1c-3c2f-4fe5-817a-1e5bc7199fb4 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:16.509509 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25089a1c-3c2f-4fe5-817a-1e5bc7199fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:16.509776 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25089a1c-3c2f-4fe5-817a-1e5bc7199fb4 None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:16.510534 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25089a1c-3c2f-4fe5-817a-1e5bc7199fb4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:16.511172 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25089a1c-3c2f-4fe5-817a-1e5bc7199fb4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:16.512581 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25089a1c-3c2f-4fe5-817a-1e5bc7199fb4 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:16.525332 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 304/589] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 05:06:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:16.526182 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25089a1c-3c2f-4fe5-817a-1e5bc7199fb4 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:16.538297 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:16.545383 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c391d-a635-4e8e-b618-c1155eb6177d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:16.545994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c391d-a635-4e8e-b618-c1155eb6177d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:16.546302 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c391d-a635-4e8e-b618-c1155eb6177d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:16.546572 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c391d-a635-4e8e-b618-c1155eb6177d None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:16.547229 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5c391d-a635-4e8e-b618-c1155eb6177d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:16.547525 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5c391d-a635-4e8e-b618-c1155eb6177d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:16.548896 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5c391d-a635-4e8e-b618-c1155eb6177d None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:16.560743 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 286/590] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:16.561826 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c391d-a635-4e8e-b618-c1155eb6177d None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:16.644170 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25089a1c-3c2f-4fe5-817a-1e5bc7199fb4 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:16.663359 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:16.671226 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c391d-a635-4e8e-b618-c1155eb6177d None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:16.680823 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0633c592-3960-4eaa-a15e-a43335feee8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:16.681457 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633c592-3960-4eaa-a15e-a43335feee8a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:16.681731 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633c592-3960-4eaa-a15e-a43335feee8a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:16.681998 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633c592-3960-4eaa-a15e-a43335feee8a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:16.689455 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0633c592-3960-4eaa-a15e-a43335feee8a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:16.690929 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0633c592-3960-4eaa-a15e-a43335feee8a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:16.702443 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-0633c592-3960-4eaa-a15e-a43335feee8a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 05:06:16.702443 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:16.702443 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:16.702443 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:16.702443 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:16.702443 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:16.702443 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:16.702443 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:16.702443 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:16.702443 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:16.702443 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:16.702443 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:16.702443 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 05:06:16.702443 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 05:06:16.702443 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 05:06:16.702443 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 05:06:16.703392 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:16.703392 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:16.703392 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 05:06:16.703392 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:16.703392 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 05:06:16.703392 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:16.703392 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:16.703392 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:16.703392 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:16.703392 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:16.703392 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:16.703392 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:16.703392 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:16.703392 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:16.703392 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:16.703392 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:16.703392 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:16.703392 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:16.704335 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:16.704335 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:16.704335 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 05:06:16.704335 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 05:06:16.704335 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 05:06:16.704335 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 05:06:16.704335 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 05:06:16.704335 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 05:06:16.704335 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 05:06:16.704335 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:16.708136 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 305/591] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:06:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:16.708755 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0633c592-3960-4eaa-a15e-a43335feee8a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:16.708936 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b5780a-132d-4da9-893d-8de7b23ec3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:16.709552 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5780a-132d-4da9-893d-8de7b23ec3ae None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:16.709833 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5780a-132d-4da9-893d-8de7b23ec3ae None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:16.710137 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5780a-132d-4da9-893d-8de7b23ec3ae None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:16.710835 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b5780a-132d-4da9-893d-8de7b23ec3ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:16.711237 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b5780a-132d-4da9-893d-8de7b23ec3ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:16.713061 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b5780a-132d-4da9-893d-8de7b23ec3ae None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:16.725493 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 287/592] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 05:06:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:16.726271 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b5780a-132d-4da9-893d-8de7b23ec3ae None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:16.782357 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:16.820213 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0633c592-3960-4eaa-a15e-a43335feee8a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:16.844553 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b5780a-132d-4da9-893d-8de7b23ec3ae None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:16.856017 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0785ad95-b35b-4425-a6aa-a62521ea6310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:16.856686 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785ad95-b35b-4425-a6aa-a62521ea6310 None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:16.856950 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785ad95-b35b-4425-a6aa-a62521ea6310 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:16.857214 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785ad95-b35b-4425-a6aa-a62521ea6310 None admin] PATH_INFO: `/v3/projects/e01ce1b1282e44a39d659b87f120995e/users/c08e6024b0c8420aa274653c2a916edd/roles/207bf98e31774cc0b21249de6d96155b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:16.873047 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0785ad95-b35b-4425-a6aa-a62521ea6310 None admin] RBAC: Authorizing `identity:create_grant(project_id=e01ce1b1282e44a39d659b87f120995e, user_id=c08e6024b0c8420aa274653c2a916edd, role_id=207bf98e31774cc0b21249de6d96155b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:16.874662 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0785ad95-b35b-4425-a6aa-a62521ea6310 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:16.882674 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a3b21-fa38-4f25-8d1c-1a3a48c1ed09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:16.883764 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a3b21-fa38-4f25-8d1c-1a3a48c1ed09 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:16.884048 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a3b21-fa38-4f25-8d1c-1a3a48c1ed09 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:16.884440 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a3b21-fa38-4f25-8d1c-1a3a48c1ed09 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:16.890213 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 306/593] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 05:06:16 2022] PUT /identity/v3/projects/e01ce1b1282e44a39d659b87f120995e/users/c08e6024b0c8420aa274653c2a916edd/roles/207bf98e31774cc0b21249de6d96155b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:06:16.891038 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0785ad95-b35b-4425-a6aa-a62521ea6310 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:16.892608 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8a3b21-fa38-4f25-8d1c-1a3a48c1ed09 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:16.894229 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8a3b21-fa38-4f25-8d1c-1a3a48c1ed09 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:16.913531 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-5c8a3b21-fa38-4f25-8d1c-1a3a48c1ed09 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 05:06:16.913531 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:16.913531 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:16.913531 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:16.913531 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:16.913531 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:16.913531 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:16.913531 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:16.913531 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:16.913531 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:16.913531 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:16.913531 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:16.913531 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 05:06:16.913531 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 05:06:16.913531 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 05:06:16.913531 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 05:06:16.914627 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:16.914627 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:16.914627 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 05:06:16.914627 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:16.914627 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 05:06:16.914627 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:16.914627 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:16.914627 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:16.914627 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:16.914627 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:16.914627 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:16.914627 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:16.914627 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:16.914627 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:16.914627 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:16.914627 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:16.914627 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:16.914627 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:16.915577 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:16.915577 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:16.915577 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 05:06:16.915577 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 05:06:16.915577 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 05:06:16.915577 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 05:06:16.915577 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 05:06:16.915577 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 05:06:16.915577 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 05:06:16.915577 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:16.918583 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 288/594] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 05:06:16 2022] GET /identity/v3/users/neutron => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:16.919683 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a3b21-fa38-4f25-8d1c-1a3a48c1ed09 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:16.938784 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:17.039144 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a3b21-fa38-4f25-8d1c-1a3a48c1ed09 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:17.039620 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0785ad95-b35b-4425-a6aa-a62521ea6310 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:17.064221 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:17.082322 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ebaa90-e564-441b-9b6e-e27e92c59c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:17.082941 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebaa90-e564-441b-9b6e-e27e92c59c79 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:17.083223 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebaa90-e564-441b-9b6e-e27e92c59c79 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:17.083489 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebaa90-e564-441b-9b6e-e27e92c59c79 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:17.084226 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ebaa90-e564-441b-9b6e-e27e92c59c79 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:17.084527 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ebaa90-e564-441b-9b6e-e27e92c59c79 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:17.084840 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af203acc-e612-4074-8c78-d1c87eb51623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:17.085592 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af203acc-e612-4074-8c78-d1c87eb51623 None admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:17.085954 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ebaa90-e564-441b-9b6e-e27e92c59c79 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:17.086122 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af203acc-e612-4074-8c78-d1c87eb51623 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:17.086446 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af203acc-e612-4074-8c78-d1c87eb51623 None admin] PATH_INFO: `/v3/projects/e01ce1b1282e44a39d659b87f120995e/users/f27665b1574a4db7b7a9771f7d775826/roles/207bf98e31774cc0b21249de6d96155b` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:17.104605 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af203acc-e612-4074-8c78-d1c87eb51623 None admin] RBAC: Authorizing `identity:create_grant(project_id=e01ce1b1282e44a39d659b87f120995e, user_id=f27665b1574a4db7b7a9771f7d775826, role_id=207bf98e31774cc0b21249de6d96155b)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:17.104988 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 307/595] 10.222.0.36 () {56 vars in 1137 bytes} [Wed Aug 31 05:06:16 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:17.105824 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ebaa90-e564-441b-9b6e-e27e92c59c79 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:17.106456 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af203acc-e612-4074-8c78-d1c87eb51623 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:17.126249 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 289/596] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 05:06:16 2022] PUT /identity/v3/projects/e01ce1b1282e44a39d659b87f120995e/users/f27665b1574a4db7b7a9771f7d775826/roles/207bf98e31774cc0b21249de6d96155b => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:06:17.127709 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af203acc-e612-4074-8c78-d1c87eb51623 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:17.237303 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:17.253419 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ebaa90-e564-441b-9b6e-e27e92c59c79 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:17.305994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af203acc-e612-4074-8c78-d1c87eb51623 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:17.320994 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a4e0a7-bb61-4ba1-b25c-dd7a036816a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:17.321612 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4e0a7-bb61-4ba1-b25c-dd7a036816a4 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:17.321887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4e0a7-bb61-4ba1-b25c-dd7a036816a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:17.322193 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4e0a7-bb61-4ba1-b25c-dd7a036816a4 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:17.322935 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a4e0a7-bb61-4ba1-b25c-dd7a036816a4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:17.323232 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a4e0a7-bb61-4ba1-b25c-dd7a036816a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:17.324656 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a4e0a7-bb61-4ba1-b25c-dd7a036816a4 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:17.346597 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa23ea-9373-4e64-950d-56c40a0d8cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:17.347235 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa23ea-9373-4e64-950d-56c40a0d8cf9 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:17.347516 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa23ea-9373-4e64-950d-56c40a0d8cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:17.347788 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa23ea-9373-4e64-950d-56c40a0d8cf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:17.347905 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 308/597] 10.222.0.36 () {56 vars in 1139 bytes} [Wed Aug 31 05:06:17 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:17.354567 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a4e0a7-bb61-4ba1-b25c-dd7a036816a4 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:17.361368 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa23ea-9373-4e64-950d-56c40a0d8cf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:17.362947 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa23ea-9373-4e64-950d-56c40a0d8cf9 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:17.380653 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-f2aa23ea-9373-4e64-950d-56c40a0d8cf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:17.380653 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:17.380653 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:17.380653 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:17.380653 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:17.380653 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:17.380653 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:17.380653 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:17.380653 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:17.380653 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:17.380653 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:17.380653 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:17.380653 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:17.380653 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:17.380653 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:17.380653 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:17.381938 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:17.381938 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:17.381938 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:17.381938 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:17.381938 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:17.381938 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:17.381938 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:17.381938 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:17.381938 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:17.381938 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:17.381938 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:17.381938 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:17.381938 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:17.381938 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:17.381938 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:17.381938 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:17.381938 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:17.382895 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:17.382895 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:17.382895 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:17.382895 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 290/598] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:17.388010 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa23ea-9373-4e64-950d-56c40a0d8cf9 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:17.402379 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:17.475242 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a4e0a7-bb61-4ba1-b25c-dd7a036816a4 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:17.542466 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169885c9-f4e2-41cb-a3d5-b8c1d500829c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:17.542720 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa23ea-9373-4e64-950d-56c40a0d8cf9 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:17.543213 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169885c9-f4e2-41cb-a3d5-b8c1d500829c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:17.543496 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169885c9-f4e2-41cb-a3d5-b8c1d500829c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:17.543765 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169885c9-f4e2-41cb-a3d5-b8c1d500829c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:17.560174 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:17.578400 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169885c9-f4e2-41cb-a3d5-b8c1d500829c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:17.579877 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169885c9-f4e2-41cb-a3d5-b8c1d500829c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:17.580180 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab92a1-39db-4dd7-8d45-380bb2368f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:17.580878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab92a1-39db-4dd7-8d45-380bb2368f49 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:17.581155 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab92a1-39db-4dd7-8d45-380bb2368f49 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:17.581423 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab92a1-39db-4dd7-8d45-380bb2368f49 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:17.582086 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aab92a1-39db-4dd7-8d45-380bb2368f49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:17.582480 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aab92a1-39db-4dd7-8d45-380bb2368f49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:17.583867 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aab92a1-39db-4dd7-8d45-380bb2368f49 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:17.602086 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 291/599] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:17.603335 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab92a1-39db-4dd7-8d45-380bb2368f49 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:17.608768 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-169885c9-f4e2-41cb-a3d5-b8c1d500829c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:17.608768 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:17.608768 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:17.608768 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:17.608768 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:17.608768 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:17.608768 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:17.608768 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:17.608768 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:17.608768 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:17.608768 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:17.608768 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:17.608768 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:17.608768 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:17.608768 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:17.608768 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:17.609718 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:17.609718 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:17.609718 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:17.609718 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:17.609718 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:17.609718 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:17.609718 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:17.609718 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:17.609718 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:17.609718 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:17.609718 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:17.609718 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:17.609718 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:17.609718 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:17.609718 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:17.609718 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:17.609718 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:17.610663 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:17.610663 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:17.610663 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:17.610663 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 309/600] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:17.617478 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169885c9-f4e2-41cb-a3d5-b8c1d500829c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:17.750966 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab92a1-39db-4dd7-8d45-380bb2368f49 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:17.788324 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fafd603-b091-4c65-9a9a-9d559eb63f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:17.788953 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fafd603-b091-4c65-9a9a-9d559eb63f17 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:17.789329 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fafd603-b091-4c65-9a9a-9d559eb63f17 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:17.789604 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fafd603-b091-4c65-9a9a-9d559eb63f17 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:17.797171 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fafd603-b091-4c65-9a9a-9d559eb63f17 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:17.798624 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fafd603-b091-4c65-9a9a-9d559eb63f17 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:17.799889 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:17.813919 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-3fafd603-b091-4c65-9a9a-9d559eb63f17 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:17.813919 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:17.813919 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:17.813919 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:17.813919 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:17.813919 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:17.813919 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:17.813919 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:17.813919 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:17.813919 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:17.813919 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:17.813919 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:17.813919 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 05:06:17.813919 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 05:06:17.813919 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 05:06:17.814869 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 05:06:17.814869 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:17.814869 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:17.814869 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:17.814869 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:17.814869 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:17.814869 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:17.814869 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:17.814869 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:17.814869 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:17.814869 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:17.814869 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:17.814869 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:17.814869 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:17.814869 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:17.814869 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:17.814869 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:17.815769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 05:06:17.815769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 05:06:17.815769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 05:06:17.815769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 05:06:17.815769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 05:06:17.815769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 05:06:17.815769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:17.815769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:17.815769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 292/601] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:06:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:06:17.820788 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fafd603-b091-4c65-9a9a-9d559eb63f17 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:17.839450 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169885c9-f4e2-41cb-a3d5-b8c1d500829c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:17.886611 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ceea8c-d044-43c8-8929-05819393ceed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:17.887231 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ceea8c-d044-43c8-8929-05819393ceed None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:17.887511 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ceea8c-d044-43c8-8929-05819393ceed None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:17.887844 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ceea8c-d044-43c8-8929-05819393ceed None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:17.888562 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ceea8c-d044-43c8-8929-05819393ceed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:17.889018 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ceea8c-d044-43c8-8929-05819393ceed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:17.890711 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ceea8c-d044-43c8-8929-05819393ceed None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:17.906037 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:17.910190 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 310/602] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:17.913138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ceea8c-d044-43c8-8929-05819393ceed None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:18.048932 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fafd603-b091-4c65-9a9a-9d559eb63f17 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:18.052355 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ceea8c-d044-43c8-8929-05819393ceed None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:18.093753 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19f30a-d84a-4c4b-9347-2cfbb35fb15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:18.094438 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19f30a-d84a-4c4b-9347-2cfbb35fb15d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:18.094718 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19f30a-d84a-4c4b-9347-2cfbb35fb15d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:18.094989 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19f30a-d84a-4c4b-9347-2cfbb35fb15d None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:18.095687 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c19f30a-d84a-4c4b-9347-2cfbb35fb15d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:18.095990 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c19f30a-d84a-4c4b-9347-2cfbb35fb15d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:18.097437 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c19f30a-d84a-4c4b-9347-2cfbb35fb15d None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:18.125881 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 293/603] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 05:06:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:18.129283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf47759d-e760-4ea3-a87e-5ea7754991cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:18.129974 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf47759d-e760-4ea3-a87e-5ea7754991cf None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:18.130294 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf47759d-e760-4ea3-a87e-5ea7754991cf None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:18.130566 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf47759d-e760-4ea3-a87e-5ea7754991cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:18.131946 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19f30a-d84a-4c4b-9347-2cfbb35fb15d None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:18.149839 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf47759d-e760-4ea3-a87e-5ea7754991cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:18.151342 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf47759d-e760-4ea3-a87e-5ea7754991cf None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:18.171002 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-cf47759d-e760-4ea3-a87e-5ea7754991cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:18.171002 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:18.171002 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:18.171002 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:18.171002 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:18.171002 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:18.171002 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:18.171002 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:18.171002 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:18.171002 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:18.171002 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:18.171002 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:18.171002 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 05:06:18.171002 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 05:06:18.171002 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 05:06:18.173011 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 05:06:18.173011 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:18.173011 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:18.173011 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:18.173011 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:18.173011 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:18.173011 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:18.173011 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:18.173011 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:18.173011 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:18.173011 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:18.173011 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:18.173011 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:18.173011 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:18.173011 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:18.173011 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:18.173011 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:18.175075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 05:06:18.175075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 05:06:18.175075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 05:06:18.175075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 05:06:18.175075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 05:06:18.175075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 05:06:18.175075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:18.175075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:18.178473 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 311/604] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:06:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:18.183952 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf47759d-e760-4ea3-a87e-5ea7754991cf None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:18.203052 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:18.247105 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19f30a-d84a-4c4b-9347-2cfbb35fb15d None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:18.250162 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:18.283683 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b87ebb3-1b16-4b46-ae6b-25b377113da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:18.284350 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87ebb3-1b16-4b46-ae6b-25b377113da8 None admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:18.284631 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87ebb3-1b16-4b46-ae6b-25b377113da8 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:18.284908 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87ebb3-1b16-4b46-ae6b-25b377113da8 None admin] PATH_INFO: `/v3/projects/e01ce1b1282e44a39d659b87f120995e/users/a8300075371341a6b9cd4dcf8ec7fa8f/roles/207bf98e31774cc0b21249de6d96155b` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:18.301226 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b87ebb3-1b16-4b46-ae6b-25b377113da8 None admin] RBAC: Authorizing `identity:create_grant(project_id=e01ce1b1282e44a39d659b87f120995e, user_id=a8300075371341a6b9cd4dcf8ec7fa8f, role_id=207bf98e31774cc0b21249de6d96155b)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:18.302652 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b87ebb3-1b16-4b46-ae6b-25b377113da8 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:18.320428 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 294/605] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 05:06:18 2022] PUT /identity/v3/projects/e01ce1b1282e44a39d659b87f120995e/users/a8300075371341a6b9cd4dcf8ec7fa8f/roles/207bf98e31774cc0b21249de6d96155b => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:06:18.370155 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483ba80-ede8-471d-8d6a-61a93e43acf0 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:18.370941 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483ba80-ede8-471d-8d6a-61a93e43acf0 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:18.371316 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483ba80-ede8-471d-8d6a-61a93e43acf0 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:18.373969 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 295/606] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 05:06:18.379054 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a116ea-0898-4bdc-b307-3f741940ba6b None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:18.379332 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a116ea-0898-4bdc-b307-3f741940ba6b None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:18.379603 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a116ea-0898-4bdc-b307-3f741940ba6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:18.401479 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-66a116ea-0898-4bdc-b307-3f741940ba6b None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:18.403452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-66a116ea-0898-4bdc-b307-3f741940ba6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:18.408378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf47759d-e760-4ea3-a87e-5ea7754991cf None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:18.484893 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 296/607] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 05:06:18.511095 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc6de12-f568-4fd6-91ba-126bab0a5b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:18.511816 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc6de12-f568-4fd6-91ba-126bab0a5b2c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:18.512099 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc6de12-f568-4fd6-91ba-126bab0a5b2c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:18.512375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc6de12-f568-4fd6-91ba-126bab0a5b2c None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:18.513095 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc6de12-f568-4fd6-91ba-126bab0a5b2c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:18.513451 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc6de12-f568-4fd6-91ba-126bab0a5b2c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:18.515166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc6de12-f568-4fd6-91ba-126bab0a5b2c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:18.539272 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 312/608] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 05:06:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:18.545621 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a116ea-0898-4bdc-b307-3f741940ba6b None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:18.625371 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:18.649322 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a116ea-0898-4bdc-b307-3f741940ba6b None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:18.685952 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32835f14-e9ba-496f-aee9-095841b3b4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:18.686679 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32835f14-e9ba-496f-aee9-095841b3b4ac None admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:18.687228 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32835f14-e9ba-496f-aee9-095841b3b4ac None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:18.687599 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32835f14-e9ba-496f-aee9-095841b3b4ac None admin] PATH_INFO: `/v3/projects/e01ce1b1282e44a39d659b87f120995e/users/090e1409506542f6831e1f65d44aae74/roles/207bf98e31774cc0b21249de6d96155b` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:18.702285 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32835f14-e9ba-496f-aee9-095841b3b4ac None admin] RBAC: Authorizing `identity:create_grant(project_id=e01ce1b1282e44a39d659b87f120995e, user_id=090e1409506542f6831e1f65d44aae74, role_id=207bf98e31774cc0b21249de6d96155b)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:18.703762 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32835f14-e9ba-496f-aee9-095841b3b4ac None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:18.718852 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 297/609] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 05:06:18 2022] PUT /identity/v3/projects/e01ce1b1282e44a39d659b87f120995e/users/090e1409506542f6831e1f65d44aae74/roles/207bf98e31774cc0b21249de6d96155b => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:06:18.878963 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc6de12-f568-4fd6-91ba-126bab0a5b2c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:18.949391 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:18.987031 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc6de12-f568-4fd6-91ba-126bab0a5b2c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:19.022981 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f907618-a74b-43ba-bd2c-b91b413904f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:19.023610 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f907618-a74b-43ba-bd2c-b91b413904f1 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:19.023886 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f907618-a74b-43ba-bd2c-b91b413904f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:19.024152 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f907618-a74b-43ba-bd2c-b91b413904f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:19.044861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f907618-a74b-43ba-bd2c-b91b413904f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:06:19.049765 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-3f907618-a74b-43ba-bd2c-b91b413904f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 05:06:19.049765 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:19.049765 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:19.049765 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:19.049765 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:19.049765 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:19.049765 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:19.049765 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:19.049765 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:19.049765 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:19.049765 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:19.049765 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:19.049765 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:06:19.049765 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:06:19.049765 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:06:19.049765 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:06:19.050735 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:06:19.050735 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:06:19.050735 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:06:19.050735 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:06:19.050735 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:19.050735 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:19.050735 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:19.050735 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:19.050735 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:19.050735 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:19.050735 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:19.050735 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:19.050735 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:19.050735 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:19.050735 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:19.050735 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:19.051592 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:19.051592 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:19.051592 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:19.051592 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:19.051592 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:06:19.051592 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:06:19.051592 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:06:19.051592 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:06:19.051592 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:06:19.051592 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:06:19.051592 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 05:06:19.051592 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:19.051592 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 313/610] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 05:06:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:06:19.057252 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32835f14-e9ba-496f-aee9-095841b3b4ac None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:19.211137 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32835f14-e9ba-496f-aee9-095841b3b4ac None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:19.242996 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdefb007-bc29-445f-b4f6-131103b47bd1 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:19.243629 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdefb007-bc29-445f-b4f6-131103b47bd1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:19.243858 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdefb007-bc29-445f-b4f6-131103b47bd1 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:19.245450 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 314/611] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:19.247857 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:19.255473 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559a3c8-971c-4aa5-bf8d-a496b0f1b04d None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:19.255898 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559a3c8-971c-4aa5-bf8d-a496b0f1b04d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:19.256177 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559a3c8-971c-4aa5-bf8d-a496b0f1b04d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:19.258279 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b5433-208f-48b3-933c-1b9ac21dc113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:19.258890 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b5433-208f-48b3-933c-1b9ac21dc113 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:19.259168 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b5433-208f-48b3-933c-1b9ac21dc113 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:19.259529 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b5433-208f-48b3-933c-1b9ac21dc113 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:19.260203 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194b5433-208f-48b3-933c-1b9ac21dc113 None admin] RBAC: Adding query filter params (name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:19.260499 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194b5433-208f-48b3-933c-1b9ac21dc113 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:19.261865 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194b5433-208f-48b3-933c-1b9ac21dc113 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:19.271412 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 298/612] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Aug 31 05:06:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:19.281279 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b5433-208f-48b3-933c-1b9ac21dc113 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:19.289525 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e559a3c8-971c-4aa5-bf8d-a496b0f1b04d None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:19.294965 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e559a3c8-971c-4aa5-bf8d-a496b0f1b04d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:19.414696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 315/613] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:06:19.415991 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ddd27-8548-4502-8d50-4fd74322d20f None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:19.416280 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ddd27-8548-4502-8d50-4fd74322d20f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:19.416559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ddd27-8548-4502-8d50-4fd74322d20f None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:19.418595 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 316/614] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:19.427525 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75032ee9-9b2b-43ea-9e01-62969646ea0a None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:19.427814 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75032ee9-9b2b-43ea-9e01-62969646ea0a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:19.428087 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75032ee9-9b2b-43ea-9e01-62969646ea0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:19.447692 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-75032ee9-9b2b-43ea-9e01-62969646ea0a None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:19.448894 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-75032ee9-9b2b-43ea-9e01-62969646ea0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:19.515230 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:19.520591 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b5433-208f-48b3-933c-1b9ac21dc113 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:19.521746 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 317/615] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:06:19.584255 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b7491-6fb3-4665-98f6-c9013e59bc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:19.584891 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b7491-6fb3-4665-98f6-c9013e59bc6b None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:19.585168 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b7491-6fb3-4665-98f6-c9013e59bc6b None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:19.585436 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b7491-6fb3-4665-98f6-c9013e59bc6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:19.596579 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9b7491-6fb3-4665-98f6-c9013e59bc6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:19.598070 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9b7491-6fb3-4665-98f6-c9013e59bc6b None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:19.616010 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-7e9b7491-6fb3-4665-98f6-c9013e59bc6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:19.616010 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:19.616010 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:19.616010 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:19.616010 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:19.616010 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:19.616010 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:19.616010 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:19.616010 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:19.616010 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:19.616010 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:19.616010 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:19.616010 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:19.616010 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:19.616010 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:19.616010 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:19.616958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:19.616958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:19.616958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:19.616958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:19.616958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:19.616958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:19.616958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:19.616958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:19.616958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:19.616958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:19.616958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:19.616958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:19.616958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:19.616958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:19.616958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:19.616958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:19.616958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:19.617856 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:19.617856 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:19.617856 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:19.617856 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 299/616] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:06:19.623126 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75032ee9-9b2b-43ea-9e01-62969646ea0a None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:19.720409 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b7491-6fb3-4665-98f6-c9013e59bc6b None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:19.770690 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:19.828028 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75032ee9-9b2b-43ea-9e01-62969646ea0a None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:19.839004 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b7491-6fb3-4665-98f6-c9013e59bc6b None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:19.879718 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f820bd72-c61b-4a32-9c66-9145bbffa978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:19.880356 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f820bd72-c61b-4a32-9c66-9145bbffa978 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:19.880637 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f820bd72-c61b-4a32-9c66-9145bbffa978 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:19.880910 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f820bd72-c61b-4a32-9c66-9145bbffa978 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:19.884773 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791b7ded-03f1-4d82-92b0-846a95dc47e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:19.885620 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791b7ded-03f1-4d82-92b0-846a95dc47e3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:19.886151 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791b7ded-03f1-4d82-92b0-846a95dc47e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:19.886614 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791b7ded-03f1-4d82-92b0-846a95dc47e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:19.887453 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791b7ded-03f1-4d82-92b0-846a95dc47e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:19.887948 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791b7ded-03f1-4d82-92b0-846a95dc47e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:19.889594 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791b7ded-03f1-4d82-92b0-846a95dc47e3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:19.891967 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f820bd72-c61b-4a32-9c66-9145bbffa978 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:06:19.897647 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-f820bd72-c61b-4a32-9c66-9145bbffa978 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 05:06:19.897647 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:19.897647 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:19.897647 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:19.897647 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:19.897647 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:19.897647 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:19.897647 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:19.897647 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:19.897647 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:19.897647 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:19.897647 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:19.897647 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:06:19.897647 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:06:19.897647 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:06:19.897647 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:06:19.898769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:06:19.898769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:06:19.898769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:06:19.898769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:06:19.898769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:19.898769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:19.898769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:19.898769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:19.898769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:19.898769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:19.898769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:19.898769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:19.898769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:19.898769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:19.898769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:19.898769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:19.899613 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:19.899613 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:19.899613 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:19.899613 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:19.899613 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:06:19.899613 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:06:19.899613 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:06:19.899613 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:06:19.899613 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:06:19.899613 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:06:19.899613 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 05:06:19.899613 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:19.899613 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 300/617] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 05:06:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:19.900347 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f820bd72-c61b-4a32-9c66-9145bbffa978 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:19.910041 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 318/618] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:19.911265 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791b7ded-03f1-4d82-92b0-846a95dc47e3 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:19.965167 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:20.025595 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:20.029874 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f820bd72-c61b-4a32-9c66-9145bbffa978 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:20.048797 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791b7ded-03f1-4d82-92b0-846a95dc47e3 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:20.066763 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf50ef-f1d0-4171-bfc3-bff7d467d593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:20.067741 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf50ef-f1d0-4171-bfc3-bff7d467d593 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:20.068254 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf50ef-f1d0-4171-bfc3-bff7d467d593 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:20.068727 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf50ef-f1d0-4171-bfc3-bff7d467d593 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:20.082209 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbf50ef-f1d0-4171-bfc3-bff7d467d593 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:06:20.088550 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10000f6b-4697-4bbd-98ab-c6b823d75f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:20.089164 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10000f6b-4697-4bbd-98ab-c6b823d75f4a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:20.089445 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10000f6b-4697-4bbd-98ab-c6b823d75f4a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:20.089717 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10000f6b-4697-4bbd-98ab-c6b823d75f4a None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:20.089972 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-5bbf50ef-f1d0-4171-bfc3-bff7d467d593 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 05:06:20.089972 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:20.089972 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:20.089972 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:20.089972 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:20.089972 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:20.089972 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:20.089972 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:20.089972 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:20.089972 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:20.089972 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:20.089972 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:20.089972 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:06:20.089972 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:06:20.089972 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:06:20.089972 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:06:20.090996 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:06:20.090996 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:06:20.090996 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:06:20.090996 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:06:20.090996 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:20.090996 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:20.090996 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:20.090996 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:20.090996 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:20.090996 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:20.090996 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:20.090996 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:20.090996 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:20.090996 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:20.090996 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:20.090996 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:20.091889 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:20.091889 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:20.091889 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:20.091889 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:20.091889 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:06:20.091889 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:06:20.091889 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:06:20.091889 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:06:20.091889 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:06:20.091889 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:06:20.091889 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 05:06:20.091889 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:20.092535 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10000f6b-4697-4bbd-98ab-c6b823d75f4a None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:20.092535 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10000f6b-4697-4bbd-98ab-c6b823d75f4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:20.092535 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10000f6b-4697-4bbd-98ab-c6b823d75f4a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:20.099867 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 301/619] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 05:06:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:20.099867 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf50ef-f1d0-4171-bfc3-bff7d467d593 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:20.106543 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 319/620] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Aug 31 05:06:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:20.107606 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10000f6b-4697-4bbd-98ab-c6b823d75f4a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:20.219679 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10000f6b-4697-4bbd-98ab-c6b823d75f4a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:20.229777 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:20.242859 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf50ef-f1d0-4171-bfc3-bff7d467d593 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:20.268770 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b9a591-e1d0-4cf6-a155-8f586fd3dc4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:20.270027 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b9a591-e1d0-4cf6-a155-8f586fd3dc4c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:20.270510 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b9a591-e1d0-4cf6-a155-8f586fd3dc4c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:20.271449 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b9a591-e1d0-4cf6-a155-8f586fd3dc4c None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:20.272606 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b9a591-e1d0-4cf6-a155-8f586fd3dc4c None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:20.273521 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b9a591-e1d0-4cf6-a155-8f586fd3dc4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:20.275567 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b9a591-e1d0-4cf6-a155-8f586fd3dc4c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:20.291460 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 320/621] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Aug 31 05:06:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:20.292636 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b9a591-e1d0-4cf6-a155-8f586fd3dc4c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:20.299757 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25095062-bb31-49a8-b9df-eaaace90b08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:20.300416 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25095062-bb31-49a8-b9df-eaaace90b08d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:20.300691 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25095062-bb31-49a8-b9df-eaaace90b08d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:20.300958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25095062-bb31-49a8-b9df-eaaace90b08d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:20.315628 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25095062-bb31-49a8-b9df-eaaace90b08d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:20.318778 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25095062-bb31-49a8-b9df-eaaace90b08d None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:20.332779 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-25095062-bb31-49a8-b9df-eaaace90b08d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 05:06:20.332779 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:20.332779 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:20.332779 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:20.332779 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:20.332779 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:20.332779 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:20.332779 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:20.332779 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:20.332779 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:20.332779 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:20.332779 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:20.332779 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 05:06:20.332779 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 05:06:20.332779 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 05:06:20.332779 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 05:06:20.333769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:20.333769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:20.333769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 05:06:20.333769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:20.333769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 05:06:20.333769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:20.333769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:20.333769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:20.333769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:20.333769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:20.333769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:20.333769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:20.333769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:20.333769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:20.333769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:20.333769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:20.333769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:20.333769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:20.334769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:20.334769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:20.334769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 05:06:20.334769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 05:06:20.334769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 05:06:20.334769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 05:06:20.334769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 05:06:20.334769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 05:06:20.334769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 05:06:20.334769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:20.338472 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 302/622] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 05:06:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:20.339346 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25095062-bb31-49a8-b9df-eaaace90b08d None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:20.378480 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:20.444943 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b9a591-e1d0-4cf6-a155-8f586fd3dc4c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:20.484358 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42290c3-725f-48ea-ae96-1fbef115837f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:20.485349 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42290c3-725f-48ea-ae96-1fbef115837f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:20.485824 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42290c3-725f-48ea-ae96-1fbef115837f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:20.486317 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42290c3-725f-48ea-ae96-1fbef115837f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:20.499675 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42290c3-725f-48ea-ae96-1fbef115837f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:20.501706 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42290c3-725f-48ea-ae96-1fbef115837f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:20.518934 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-f42290c3-725f-48ea-ae96-1fbef115837f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:20.518934 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:20.518934 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:20.518934 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:20.518934 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:20.518934 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:20.518934 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:20.518934 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:20.518934 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:20.518934 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:20.518934 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:20.518934 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:20.518934 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:20.518934 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:20.518934 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:20.518934 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:20.520367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:20.520367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:20.520367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:20.520367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:20.520367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:20.520367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:20.520367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:20.520367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:20.520367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:20.520367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:20.520367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:20.520367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:20.520367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:20.520367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:20.520367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:20.520367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:20.520367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:20.521283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:20.521283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:20.521283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:20.523865 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 321/623] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:20.525585 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42290c3-725f-48ea-ae96-1fbef115837f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:20.527286 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25095062-bb31-49a8-b9df-eaaace90b08d None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:20.555168 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:20.573508 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa00ce5-ad56-4861-b0bf-4e5d94055dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:20.573857 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa00ce5-ad56-4861-b0bf-4e5d94055dd2 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:20.575305 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa00ce5-ad56-4861-b0bf-4e5d94055dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:20.575592 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa00ce5-ad56-4861-b0bf-4e5d94055dd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:20.593833 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa00ce5-ad56-4861-b0bf-4e5d94055dd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:20.597952 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa00ce5-ad56-4861-b0bf-4e5d94055dd2 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:20.617128 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-4aa00ce5-ad56-4861-b0bf-4e5d94055dd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:20.617128 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:20.617128 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:20.617128 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:20.617128 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:20.617128 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:20.617128 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:20.617128 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:20.617128 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:20.617128 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:20.617128 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:20.617128 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:20.617128 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:20.617128 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:20.617128 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:20.617128 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:20.618512 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:20.618512 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:20.618512 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:20.618512 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:20.618512 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:20.618512 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:20.618512 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:20.618512 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:20.618512 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:20.618512 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:20.618512 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:20.618512 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:20.618512 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:20.618512 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:20.618512 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:20.618512 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:20.618512 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:20.619423 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:20.619423 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:20.619423 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:20.625006 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 303/624] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:20.625944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa00ce5-ad56-4861-b0bf-4e5d94055dd2 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:20.649385 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42290c3-725f-48ea-ae96-1fbef115837f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:20.690086 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add91208-d54e-4a91-9be1-88a4e0c9baa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:20.691050 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add91208-d54e-4a91-9be1-88a4e0c9baa0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:20.691524 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add91208-d54e-4a91-9be1-88a4e0c9baa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:20.691984 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add91208-d54e-4a91-9be1-88a4e0c9baa0 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:20.692954 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add91208-d54e-4a91-9be1-88a4e0c9baa0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:20.693467 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add91208-d54e-4a91-9be1-88a4e0c9baa0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:20.695138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add91208-d54e-4a91-9be1-88a4e0c9baa0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:20.713515 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 322/625] 10.222.0.36 () {56 vars in 1135 bytes} [Wed Aug 31 05:06:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:20.714850 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add91208-d54e-4a91-9be1-88a4e0c9baa0 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:20.730017 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:20.780905 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa00ce5-ad56-4861-b0bf-4e5d94055dd2 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:20.800176 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:20.821751 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add91208-d54e-4a91-9be1-88a4e0c9baa0 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:20.824043 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1459f7d0-da03-44b2-bab7-ced6e21aea2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:20.824591 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459f7d0-da03-44b2-bab7-ced6e21aea2e None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:20.824906 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459f7d0-da03-44b2-bab7-ced6e21aea2e None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:20.825181 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459f7d0-da03-44b2-bab7-ced6e21aea2e None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:20.826642 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1459f7d0-da03-44b2-bab7-ced6e21aea2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:20.832275 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1459f7d0-da03-44b2-bab7-ced6e21aea2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:20.833738 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1459f7d0-da03-44b2-bab7-ced6e21aea2e None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:20.860337 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc139c4-dd20-4e21-a046-0caccc61a37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:20.861326 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc139c4-dd20-4e21-a046-0caccc61a37f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:20.861800 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc139c4-dd20-4e21-a046-0caccc61a37f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:20.862308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc139c4-dd20-4e21-a046-0caccc61a37f None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:20.863143 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc139c4-dd20-4e21-a046-0caccc61a37f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:20.863655 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc139c4-dd20-4e21-a046-0caccc61a37f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:20.865274 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc139c4-dd20-4e21-a046-0caccc61a37f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:20.871137 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 304/626] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:20.872004 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1459f7d0-da03-44b2-bab7-ced6e21aea2e None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:20.878903 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 323/627] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:20.880160 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc17cf-9553-4c74-bf3f-fdde35f3decb None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:20.880438 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc17cf-9553-4c74-bf3f-fdde35f3decb None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:20.880705 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc17cf-9553-4c74-bf3f-fdde35f3decb None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:20.882032 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 324/628] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:20.884399 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177b64a-3f67-47c6-994e-703661209dd8 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:20.884399 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177b64a-3f67-47c6-994e-703661209dd8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:20.884399 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177b64a-3f67-47c6-994e-703661209dd8 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:20.884804 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 325/629] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:20.885554 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2177b64a-3f67-47c6-994e-703661209dd8 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:20.990978 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2177b64a-3f67-47c6-994e-703661209dd8 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:20.994492 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1459f7d0-da03-44b2-bab7-ced6e21aea2e None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:21.024615 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f19b50f-efbe-45aa-9ab1-fb55e047b75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:21.025517 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f19b50f-efbe-45aa-9ab1-fb55e047b75e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:21.025673 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:21.025802 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f19b50f-efbe-45aa-9ab1-fb55e047b75e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:21.026358 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f19b50f-efbe-45aa-9ab1-fb55e047b75e None admin] PATH_INFO: `/v3/users/glance` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:21.031341 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28db8b21-508f-437f-82e1-33446f4f8f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:21.031961 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db8b21-508f-437f-82e1-33446f4f8f38 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:21.032237 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db8b21-508f-437f-82e1-33446f4f8f38 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:21.032505 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db8b21-508f-437f-82e1-33446f4f8f38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:21.034291 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f19b50f-efbe-45aa-9ab1-fb55e047b75e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:21.035726 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f19b50f-efbe-45aa-9ab1-fb55e047b75e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:21.038522 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:21.044074 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28db8b21-508f-437f-82e1-33446f4f8f38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:21.045668 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28db8b21-508f-437f-82e1-33446f4f8f38 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:21.048536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-1f19b50f-efbe-45aa-9ab1-fb55e047b75e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 05:06:21.048536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:21.048536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:21.048536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:21.048536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:21.048536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:21.048536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:21.048536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:21.048536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:21.048536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:21.048536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:21.048536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:21.048536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 05:06:21.048536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 05:06:21.048536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 05:06:21.048536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 05:06:21.049612 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:21.049612 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:21.049612 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 05:06:21.049612 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:21.049612 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 05:06:21.049612 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:21.049612 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:21.049612 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:21.049612 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:21.049612 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:21.049612 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:21.049612 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:21.049612 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:21.049612 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:21.049612 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:21.049612 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:21.049612 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:21.049612 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:21.055564 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:21.055564 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:21.055564 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 05:06:21.055564 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 05:06:21.055564 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 05:06:21.055564 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 05:06:21.055564 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 05:06:21.055564 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 05:06:21.055564 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 05:06:21.055564 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:21.055564 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 326/630] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:06:20 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:21.055564 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f19b50f-efbe-45aa-9ab1-fb55e047b75e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:21.062950 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-28db8b21-508f-437f-82e1-33446f4f8f38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:21.062950 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:21.062950 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:21.062950 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:21.062950 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:21.062950 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:21.062950 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:21.062950 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:21.062950 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:21.062950 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:21.062950 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:21.062950 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:21.062950 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:21.062950 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:21.062950 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:21.062950 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:21.064926 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:21.064926 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:21.064926 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:21.064926 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:21.064926 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:21.064926 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:21.064926 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:21.064926 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:21.064926 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:21.064926 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:21.064926 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:21.064926 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:21.064926 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:21.064926 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:21.064926 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:21.064926 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:21.064926 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:21.067005 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:21.067005 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:21.067005 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:21.067005 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 305/631] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:21.067005 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679caa0c-ef9f-499c-b46e-5d85e2230e8c None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:21.067005 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679caa0c-ef9f-499c-b46e-5d85e2230e8c None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:21.067005 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679caa0c-ef9f-499c-b46e-5d85e2230e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:21.086436 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-679caa0c-ef9f-499c-b46e-5d85e2230e8c None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:21.087526 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-679caa0c-ef9f-499c-b46e-5d85e2230e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:21.152429 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f19b50f-efbe-45aa-9ab1-fb55e047b75e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:21.159175 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 306/632] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:06:21.159175 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee56c2a-2fae-47dc-838e-422cba72c0bb None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:21.159175 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee56c2a-2fae-47dc-838e-422cba72c0bb None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:21.159175 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee56c2a-2fae-47dc-838e-422cba72c0bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:21.174841 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-8ee56c2a-2fae-47dc-838e-422cba72c0bb None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:21.176438 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-8ee56c2a-2fae-47dc-838e-422cba72c0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:21.189488 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c553299-ac5e-4be9-8eec-02fb8879d135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:21.190145 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c553299-ac5e-4be9-8eec-02fb8879d135 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:21.190425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c553299-ac5e-4be9-8eec-02fb8879d135 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:21.190695 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c553299-ac5e-4be9-8eec-02fb8879d135 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:21.198086 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c553299-ac5e-4be9-8eec-02fb8879d135 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:21.199521 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c553299-ac5e-4be9-8eec-02fb8879d135 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:21.210375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-8c553299-ac5e-4be9-8eec-02fb8879d135 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 05:06:21.210375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:21.210375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:21.210375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:21.210375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:21.210375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:21.210375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:21.210375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:21.210375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:21.210375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:21.210375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:21.210375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:21.210375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 05:06:21.210375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 05:06:21.210375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 05:06:21.210375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 05:06:21.211312 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:21.211312 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:21.211312 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 05:06:21.211312 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:21.211312 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 05:06:21.211312 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:21.211312 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:21.211312 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:21.211312 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:21.211312 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:21.211312 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:21.211312 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:21.211312 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:21.211312 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:21.211312 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:21.211312 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:21.211312 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:21.211312 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:21.212278 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:21.212278 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:21.212278 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 05:06:21.212278 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 05:06:21.212278 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 05:06:21.212278 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 05:06:21.212278 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 05:06:21.212278 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 05:06:21.212278 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 05:06:21.212278 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:21.212278 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 327/633] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Aug 31 05:06:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:21.212278 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c553299-ac5e-4be9-8eec-02fb8879d135 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:21.258038 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 307/634] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:06:21.259275 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee56c2a-2fae-47dc-838e-422cba72c0bb None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:21.276425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:21.322319 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c553299-ac5e-4be9-8eec-02fb8879d135 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:21.328588 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:21.361652 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671738e2-883b-4fdf-88ae-b2f7207851b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:21.362576 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671738e2-883b-4fdf-88ae-b2f7207851b0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:21.363035 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671738e2-883b-4fdf-88ae-b2f7207851b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:21.363505 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671738e2-883b-4fdf-88ae-b2f7207851b0 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:21.364413 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671738e2-883b-4fdf-88ae-b2f7207851b0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:21.364885 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671738e2-883b-4fdf-88ae-b2f7207851b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:21.366507 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671738e2-883b-4fdf-88ae-b2f7207851b0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:21.385207 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 328/635] 10.222.0.36 () {56 vars in 1137 bytes} [Wed Aug 31 05:06:21 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:21.386540 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671738e2-883b-4fdf-88ae-b2f7207851b0 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:21.396850 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee56c2a-2fae-47dc-838e-422cba72c0bb None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:21.440101 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15adcc-b787-41ee-8b2d-eedb5ea0b9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:21.440703 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15adcc-b787-41ee-8b2d-eedb5ea0b9a7 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:21.440980 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15adcc-b787-41ee-8b2d-eedb5ea0b9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:21.441240 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15adcc-b787-41ee-8b2d-eedb5ea0b9a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:21.441880 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac15adcc-b787-41ee-8b2d-eedb5ea0b9a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:21.442219 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac15adcc-b787-41ee-8b2d-eedb5ea0b9a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:21.443566 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac15adcc-b787-41ee-8b2d-eedb5ea0b9a7 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:21.455434 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 308/636] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:21.456237 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15adcc-b787-41ee-8b2d-eedb5ea0b9a7 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:21.504013 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671738e2-883b-4fdf-88ae-b2f7207851b0 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:21.522921 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:21.540622 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb44d29-65b7-4745-b42a-ed3a645eec64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:21.541234 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb44d29-65b7-4745-b42a-ed3a645eec64 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:21.541509 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb44d29-65b7-4745-b42a-ed3a645eec64 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:21.541773 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb44d29-65b7-4745-b42a-ed3a645eec64 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:21.542537 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb44d29-65b7-4745-b42a-ed3a645eec64 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:21.542828 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb44d29-65b7-4745-b42a-ed3a645eec64 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:21.544254 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb44d29-65b7-4745-b42a-ed3a645eec64 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:21.556991 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15adcc-b787-41ee-8b2d-eedb5ea0b9a7 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:21.559265 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:21.561706 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 329/637] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Aug 31 05:06:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:21.563252 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb44d29-65b7-4745-b42a-ed3a645eec64 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:21.599567 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39c2fa-02cd-4793-b070-a63ee837934f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:21.600231 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39c2fa-02cd-4793-b070-a63ee837934f None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:21.600532 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39c2fa-02cd-4793-b070-a63ee837934f None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:21.600802 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39c2fa-02cd-4793-b070-a63ee837934f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:21.668948 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc39c2fa-02cd-4793-b070-a63ee837934f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:21.670502 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc39c2fa-02cd-4793-b070-a63ee837934f None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:21.674222 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb44d29-65b7-4745-b42a-ed3a645eec64 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:21.711520 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42149da3-e373-4bfe-86fe-cc77e80be9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:21.712155 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42149da3-e373-4bfe-86fe-cc77e80be9ff None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:21.712431 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42149da3-e373-4bfe-86fe-cc77e80be9ff None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:21.712703 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42149da3-e373-4bfe-86fe-cc77e80be9ff None admin] PATH_INFO: `/v3/roles/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:21.724891 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42149da3-e373-4bfe-86fe-cc77e80be9ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:06:21.729698 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-42149da3-e373-4bfe-86fe-cc77e80be9ff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 05:06:21.729698 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:21.729698 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:21.729698 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:21.729698 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:21.729698 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:21.729698 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:21.729698 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:21.729698 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:21.729698 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:21.729698 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:21.729698 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:21.729698 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:06:21.729698 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:06:21.729698 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:06:21.729698 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:06:21.730791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:06:21.730791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:06:21.730791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:06:21.730791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:06:21.730791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:21.730791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:21.730791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:21.730791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:21.730791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:21.730791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:21.730791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:21.730791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:21.730791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:21.730791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:21.730791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:21.730791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:21.731887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:21.731887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:21.731887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:21.731887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:21.731887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:06:21.731887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:06:21.731887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:06:21.731887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:06:21.731887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:06:21.731887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:06:21.731887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 05:06:21.731887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:21.731887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 330/638] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 05:06:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:21.733021 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42149da3-e373-4bfe-86fe-cc77e80be9ff None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:21.768733 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:21.775039 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 309/639] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:06:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 05:06:21.775728 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39c2fa-02cd-4793-b070-a63ee837934f None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:21.780474 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:21.834189 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42149da3-e373-4bfe-86fe-cc77e80be9ff None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:21.866341 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aba6ae-3d23-4873-add9-6fb13f76ca62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:21.866973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aba6ae-3d23-4873-add9-6fb13f76ca62 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:21.867352 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aba6ae-3d23-4873-add9-6fb13f76ca62 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:21.867619 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aba6ae-3d23-4873-add9-6fb13f76ca62 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:21.874934 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aba6ae-3d23-4873-add9-6fb13f76ca62 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:21.876375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aba6ae-3d23-4873-add9-6fb13f76ca62 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:21.877218 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39c2fa-02cd-4793-b070-a63ee837934f None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:21.887428 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-85aba6ae-3d23-4873-add9-6fb13f76ca62 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:21.887428 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:21.887428 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:21.887428 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:21.887428 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:21.887428 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:21.887428 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:21.887428 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:21.887428 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:21.887428 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:21.887428 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:21.887428 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:21.887428 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 05:06:21.887428 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 05:06:21.887428 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 05:06:21.888370 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 05:06:21.888370 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:21.888370 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:21.888370 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:21.888370 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:21.888370 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:21.888370 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:21.888370 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:21.888370 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:21.888370 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:21.888370 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:21.888370 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:21.888370 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:21.888370 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:21.888370 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:21.888370 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:21.888370 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:21.889759 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 05:06:21.889759 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 05:06:21.889759 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 05:06:21.889759 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 05:06:21.889759 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 05:06:21.889759 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 05:06:21.889759 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:21.889759 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:21.889759 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 331/640] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:06:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:21.889759 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aba6ae-3d23-4873-add9-6fb13f76ca62 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:21.914711 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797abbc5-dc04-4627-adb7-d481fa3c32a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:21.915264 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797abbc5-dc04-4627-adb7-d481fa3c32a9 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:21.915536 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797abbc5-dc04-4627-adb7-d481fa3c32a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:21.915796 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797abbc5-dc04-4627-adb7-d481fa3c32a9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:21.925664 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797abbc5-dc04-4627-adb7-d481fa3c32a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:06:21.930338 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-797abbc5-dc04-4627-adb7-d481fa3c32a9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 05:06:21.930338 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:21.930338 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:21.930338 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:21.930338 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:21.930338 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:21.930338 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:21.930338 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:21.930338 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:21.930338 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:21.930338 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:21.930338 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:21.930338 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:06:21.930338 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:06:21.930338 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:06:21.930338 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:06:21.931542 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:06:21.931542 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:06:21.931542 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:06:21.931542 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:06:21.931542 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:21.931542 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:21.931542 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:21.931542 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:21.931542 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:21.931542 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:21.931542 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:21.931542 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:21.931542 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:21.931542 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:21.931542 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:21.931542 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:21.932707 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:21.932707 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:21.932707 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:21.932707 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:21.932707 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:06:21.932707 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:06:21.932707 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:06:21.932707 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:06:21.932707 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:06:21.932707 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:06:21.932707 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 05:06:21.932707 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:21.932707 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 310/641] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 05:06:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:21.932707 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797abbc5-dc04-4627-adb7-d481fa3c32a9 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:21.985570 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:21.992168 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aba6ae-3d23-4873-add9-6fb13f76ca62 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:21.998102 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:22.024916 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a378cb-051c-48b9-ae11-a065748dc4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:22.025519 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a378cb-051c-48b9-ae11-a065748dc4b5 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:22.025796 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a378cb-051c-48b9-ae11-a065748dc4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:22.026094 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a378cb-051c-48b9-ae11-a065748dc4b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:22.034279 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797abbc5-dc04-4627-adb7-d481fa3c32a9 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:22.068686 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c0b9c-e129-4cf9-9fd3-b06107abb6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:22.069301 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c0b9c-e129-4cf9-9fd3-b06107abb6ad None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:22.069581 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c0b9c-e129-4cf9-9fd3-b06107abb6ad None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:22.069864 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c0b9c-e129-4cf9-9fd3-b06107abb6ad None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:22.071076 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156c0b9c-e129-4cf9-9fd3-b06107abb6ad None admin] RBAC: Adding query filter params (name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:22.071605 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156c0b9c-e129-4cf9-9fd3-b06107abb6ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:22.073359 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156c0b9c-e129-4cf9-9fd3-b06107abb6ad None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:22.083659 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 311/642] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Aug 31 05:06:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:22.083659 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c0b9c-e129-4cf9-9fd3-b06107abb6ad None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:22.099112 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a378cb-051c-48b9-ae11-a065748dc4b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:22.100616 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a378cb-051c-48b9-ae11-a065748dc4b5 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:22.185182 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c0b9c-e129-4cf9-9fd3-b06107abb6ad None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:22.192259 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:22.198852 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 332/643] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:06:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 05:06:22.199642 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a378cb-051c-48b9-ae11-a065748dc4b5 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:22.210346 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:22.219838 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c677131-d366-41c5-8038-0e6bc9a9a7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:22.220543 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c677131-d366-41c5-8038-0e6bc9a9a7be None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:22.220824 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c677131-d366-41c5-8038-0e6bc9a9a7be None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:22.221091 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c677131-d366-41c5-8038-0e6bc9a9a7be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:22.231396 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c677131-d366-41c5-8038-0e6bc9a9a7be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:22.232921 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c677131-d366-41c5-8038-0e6bc9a9a7be None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:22.246878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-3c677131-d366-41c5-8038-0e6bc9a9a7be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:22.246878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:22.246878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:22.246878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:22.246878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:22.246878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:22.246878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:22.246878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:22.246878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:22.246878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:22.246878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:22.246878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:22.246878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:22.246878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:22.246878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:22.246878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:22.247980 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:22.247980 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:22.247980 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:22.247980 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:22.247980 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:22.247980 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:22.247980 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:22.247980 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:22.247980 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:22.247980 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:22.247980 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:22.247980 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:22.247980 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:22.247980 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:22.247980 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:22.247980 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:22.247980 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:22.249034 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:22.249034 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:22.249034 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:22.249034 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 312/644] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:22.249034 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c677131-d366-41c5-8038-0e6bc9a9a7be None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:22.297950 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a378cb-051c-48b9-ae11-a065748dc4b5 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:22.331654 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9883562-82ab-4feb-9bcf-47b53a99a6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:22.332258 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9883562-82ab-4feb-9bcf-47b53a99a6cc None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:22.332537 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9883562-82ab-4feb-9bcf-47b53a99a6cc None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:22.332813 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9883562-82ab-4feb-9bcf-47b53a99a6cc None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:22.333498 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9883562-82ab-4feb-9bcf-47b53a99a6cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:22.333799 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9883562-82ab-4feb-9bcf-47b53a99a6cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:22.335271 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9883562-82ab-4feb-9bcf-47b53a99a6cc None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:22.346835 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 333/645] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 05:06:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:22.347386 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c677131-d366-41c5-8038-0e6bc9a9a7be None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:22.347604 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9883562-82ab-4feb-9bcf-47b53a99a6cc None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:22.381614 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc8f8ae-2551-4e8a-afdf-4537e49658df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:22.382235 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8f8ae-2551-4e8a-afdf-4537e49658df None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:22.382508 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8f8ae-2551-4e8a-afdf-4537e49658df None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:22.382779 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8f8ae-2551-4e8a-afdf-4537e49658df None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:22.383438 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc8f8ae-2551-4e8a-afdf-4537e49658df None admin] RBAC: Adding query filter params (name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:22.383733 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc8f8ae-2551-4e8a-afdf-4537e49658df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:22.385099 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc8f8ae-2551-4e8a-afdf-4537e49658df None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:22.393022 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 313/646] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Aug 31 05:06:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:22.393823 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc8f8ae-2551-4e8a-afdf-4537e49658df None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:22.407587 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:22.440258 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:22.451012 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9883562-82ab-4feb-9bcf-47b53a99a6cc None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:22.483042 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2754df-7608-4508-b44f-1ae158c4a87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:22.483660 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2754df-7608-4508-b44f-1ae158c4a87a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:22.484016 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2754df-7608-4508-b44f-1ae158c4a87a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:22.484279 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2754df-7608-4508-b44f-1ae158c4a87a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:22.495609 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2754df-7608-4508-b44f-1ae158c4a87a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:22.496705 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc8f8ae-2551-4e8a-afdf-4537e49658df None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:22.497035 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2754df-7608-4508-b44f-1ae158c4a87a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:22.510459 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-ac2754df-7608-4508-b44f-1ae158c4a87a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:22.510459 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:22.510459 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:22.510459 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:22.510459 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:22.510459 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:22.510459 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:22.510459 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:22.510459 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:22.510459 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:22.510459 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:22.510459 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:22.510459 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:22.510459 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:22.510459 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:22.510459 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:22.511414 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:22.511414 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:22.511414 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:22.511414 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:22.511414 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:22.511414 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:22.511414 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:22.511414 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:22.511414 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:22.511414 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:22.511414 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:22.511414 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:22.511414 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:22.511414 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:22.511414 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:22.511414 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:22.511414 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:22.512343 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:22.512343 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:22.512343 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:22.512343 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 334/647] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:22.512343 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2754df-7608-4508-b44f-1ae158c4a87a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:22.533884 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c990db-c4f0-49ad-8ed0-90b3cd3c563c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:22.534543 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c990db-c4f0-49ad-8ed0-90b3cd3c563c None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:22.534821 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c990db-c4f0-49ad-8ed0-90b3cd3c563c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:22.535090 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c990db-c4f0-49ad-8ed0-90b3cd3c563c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:22.545603 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c990db-c4f0-49ad-8ed0-90b3cd3c563c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:22.547111 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c990db-c4f0-49ad-8ed0-90b3cd3c563c None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:22.561022 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-01c990db-c4f0-49ad-8ed0-90b3cd3c563c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:22.561022 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:22.561022 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:22.561022 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:22.561022 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:22.561022 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:22.561022 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:22.561022 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:22.561022 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:22.561022 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:22.561022 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:22.561022 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:22.561022 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:22.561022 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:22.561022 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:22.561022 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:22.562486 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:22.562486 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:22.562486 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:22.562486 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:22.562486 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:22.562486 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:22.562486 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:22.562486 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:22.562486 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:22.562486 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:22.562486 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:22.562486 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:22.562486 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:22.562486 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:22.562486 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:22.562486 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:22.562486 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:22.563572 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:22.563572 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:22.563572 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:22.563572 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 314/648] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:22.563572 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c990db-c4f0-49ad-8ed0-90b3cd3c563c None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:22.611993 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2754df-7608-4508-b44f-1ae158c4a87a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:22.620886 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:22.646966 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e271597a-7820-4229-9ca2-3f8be11977e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:22.647581 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271597a-7820-4229-9ca2-3f8be11977e6 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:22.647861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271597a-7820-4229-9ca2-3f8be11977e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:22.648141 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271597a-7820-4229-9ca2-3f8be11977e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:22.648783 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e271597a-7820-4229-9ca2-3f8be11977e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:22.649083 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e271597a-7820-4229-9ca2-3f8be11977e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:22.650518 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e271597a-7820-4229-9ca2-3f8be11977e6 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:22.661199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:22.666490 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 335/649] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:22.666490 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e271597a-7820-4229-9ca2-3f8be11977e6 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:22.669847 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c990db-c4f0-49ad-8ed0-90b3cd3c563c None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:22.703130 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253e8ea5-8e28-4c26-814b-6e458ef007cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:22.703712 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e8ea5-8e28-4c26-814b-6e458ef007cc None admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:22.703986 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e8ea5-8e28-4c26-814b-6e458ef007cc None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:22.704249 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e8ea5-8e28-4c26-814b-6e458ef007cc None admin] PATH_INFO: `/v3/projects/e01ce1b1282e44a39d659b87f120995e/users/d9b82080a73d4c4a8ec08b0e67d6d11c/roles/207bf98e31774cc0b21249de6d96155b` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:22.718375 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253e8ea5-8e28-4c26-814b-6e458ef007cc None admin] RBAC: Authorizing `identity:create_grant(project_id=e01ce1b1282e44a39d659b87f120995e, user_id=d9b82080a73d4c4a8ec08b0e67d6d11c, role_id=207bf98e31774cc0b21249de6d96155b)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:22.719824 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253e8ea5-8e28-4c26-814b-6e458ef007cc None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:22.732954 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 315/650] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 05:06:22 2022] PUT /identity/v3/projects/e01ce1b1282e44a39d659b87f120995e/users/d9b82080a73d4c4a8ec08b0e67d6d11c/roles/207bf98e31774cc0b21249de6d96155b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:06:22.733771 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253e8ea5-8e28-4c26-814b-6e458ef007cc None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:22.770558 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e271597a-7820-4229-9ca2-3f8be11977e6 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:22.807282 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd1af86-b555-402a-9004-55c801c0a1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:22.807941 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd1af86-b555-402a-9004-55c801c0a1e5 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:22.808201 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd1af86-b555-402a-9004-55c801c0a1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:22.808470 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd1af86-b555-402a-9004-55c801c0a1e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:22.819635 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd1af86-b555-402a-9004-55c801c0a1e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:22.821139 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd1af86-b555-402a-9004-55c801c0a1e5 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:22.836737 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-0dd1af86-b555-402a-9004-55c801c0a1e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:22.836737 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:22.836737 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:22.836737 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:22.836737 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:22.836737 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:22.836737 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:22.836737 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:22.836737 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:22.836737 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:22.836737 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:22.836737 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:22.836737 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:22.836737 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:22.836737 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:22.836737 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:22.838246 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:22.838246 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:22.838246 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:22.838246 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:22.838246 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:22.838246 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:22.838246 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:22.838246 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:22.838246 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:22.838246 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:22.838246 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:22.838246 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:22.838246 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:22.838246 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:22.838246 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:22.838246 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:22.838246 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:22.839228 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:22.839228 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:22.839228 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:22.839228 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 336/651] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:22.839228 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd1af86-b555-402a-9004-55c801c0a1e5 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:22.849315 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253e8ea5-8e28-4c26-814b-6e458ef007cc None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:22.862669 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:22.893373 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d679fb-2266-425f-98b3-b02bfce5f23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:22.894283 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d679fb-2266-425f-98b3-b02bfce5f23f None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:22.894567 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d679fb-2266-425f-98b3-b02bfce5f23f None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:22.894843 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d679fb-2266-425f-98b3-b02bfce5f23f None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:22.895548 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d679fb-2266-425f-98b3-b02bfce5f23f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:22.895879 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d679fb-2266-425f-98b3-b02bfce5f23f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:22.897266 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d679fb-2266-425f-98b3-b02bfce5f23f None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:22.915465 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 316/652] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:22.915465 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d679fb-2266-425f-98b3-b02bfce5f23f None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:22.919306 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:22.948639 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd1af86-b555-402a-9004-55c801c0a1e5 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:22.982946 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1355e4ec-e2f5-481d-9c28-9f2a71fca5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:22.983501 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355e4ec-e2f5-481d-9c28-9f2a71fca5bc None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:22.983773 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355e4ec-e2f5-481d-9c28-9f2a71fca5bc None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:22.984036 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355e4ec-e2f5-481d-9c28-9f2a71fca5bc None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:22.984672 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1355e4ec-e2f5-481d-9c28-9f2a71fca5bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:22.984968 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1355e4ec-e2f5-481d-9c28-9f2a71fca5bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:22.986364 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1355e4ec-e2f5-481d-9c28-9f2a71fca5bc None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:22.998585 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 337/653] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:22.999395 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1355e4ec-e2f5-481d-9c28-9f2a71fca5bc None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:23.029370 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d679fb-2266-425f-98b3-b02bfce5f23f None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:23.064119 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c9ebba-617d-4b9f-8f41-c6d38e8ffa49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:23.064719 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9ebba-617d-4b9f-8f41-c6d38e8ffa49 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:23.064993 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9ebba-617d-4b9f-8f41-c6d38e8ffa49 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:23.065311 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9ebba-617d-4b9f-8f41-c6d38e8ffa49 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:23.072081 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c9ebba-617d-4b9f-8f41-c6d38e8ffa49 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:23.073556 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c9ebba-617d-4b9f-8f41-c6d38e8ffa49 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:23.084994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-06c9ebba-617d-4b9f-8f41-c6d38e8ffa49 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:23.084994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:23.084994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:23.084994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:23.084994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:23.084994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:23.084994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:23.084994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:23.084994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:23.084994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:23.084994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:23.084994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:23.084994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 05:06:23.084994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 05:06:23.084994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 05:06:23.086105 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 05:06:23.086105 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:23.086105 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:23.086105 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:23.086105 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:23.086105 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:23.086105 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:23.086105 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:23.086105 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:23.086105 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:23.086105 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:23.086105 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:23.086105 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:23.086105 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:23.086105 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:23.086105 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:23.086105 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:23.087039 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 05:06:23.087039 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 05:06:23.087039 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 05:06:23.087039 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 05:06:23.087039 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 05:06:23.087039 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 05:06:23.087039 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:23.087039 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:23.087039 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 317/654] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:06:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:23.087520 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c9ebba-617d-4b9f-8f41-c6d38e8ffa49 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:23.095539 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:23.116019 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1355e4ec-e2f5-481d-9c28-9f2a71fca5bc None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:23.151385 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3e98a-02d0-4212-942c-21800f4c94b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:23.151994 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3e98a-02d0-4212-942c-21800f4c94b2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:23.152244 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3e98a-02d0-4212-942c-21800f4c94b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:23.152510 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3e98a-02d0-4212-942c-21800f4c94b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:23.153189 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac3e98a-02d0-4212-942c-21800f4c94b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:23.153707 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac3e98a-02d0-4212-942c-21800f4c94b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:23.154917 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac3e98a-02d0-4212-942c-21800f4c94b2 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:23.156468 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:23.166149 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 338/655] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:23.166960 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3e98a-02d0-4212-942c-21800f4c94b2 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:23.200949 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c9ebba-617d-4b9f-8f41-c6d38e8ffa49 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:23.236347 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9e301f-7b9b-4f09-bc23-2387f65c87dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:23.236891 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e301f-7b9b-4f09-bc23-2387f65c87dd None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:23.237162 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e301f-7b9b-4f09-bc23-2387f65c87dd None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:23.237432 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e301f-7b9b-4f09-bc23-2387f65c87dd None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:23.245410 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9e301f-7b9b-4f09-bc23-2387f65c87dd None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:23.247771 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9e301f-7b9b-4f09-bc23-2387f65c87dd None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:23.259935 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-7b9e301f-7b9b-4f09-bc23-2387f65c87dd None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 05:06:23.259935 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:23.259935 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:23.259935 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:23.259935 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:23.259935 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:23.259935 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:23.259935 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:23.259935 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:23.259935 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:23.259935 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:23.259935 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:23.259935 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 05:06:23.259935 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 05:06:23.259935 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 05:06:23.259935 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 05:06:23.260892 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:23.260892 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:23.260892 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 05:06:23.260892 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:23.260892 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 05:06:23.260892 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:23.260892 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:23.260892 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:23.260892 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:23.260892 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:23.260892 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:23.260892 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:23.260892 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:23.260892 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:23.260892 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:23.260892 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:23.260892 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:23.260892 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:23.261855 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:23.261855 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:23.261855 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 05:06:23.261855 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 05:06:23.261855 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 05:06:23.261855 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 05:06:23.261855 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 05:06:23.261855 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 05:06:23.261855 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 05:06:23.261855 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:23.261855 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 318/656] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 05:06:23 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:23.261855 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9e301f-7b9b-4f09-bc23-2387f65c87dd None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:23.282662 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3e98a-02d0-4212-942c-21800f4c94b2 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:23.318704 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f967df2-1c4d-4e3b-83b3-cee99ddb0841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:23.319327 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f967df2-1c4d-4e3b-83b3-cee99ddb0841 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:23.319600 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f967df2-1c4d-4e3b-83b3-cee99ddb0841 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:23.319864 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f967df2-1c4d-4e3b-83b3-cee99ddb0841 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:23.327360 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f967df2-1c4d-4e3b-83b3-cee99ddb0841 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:23.328806 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f967df2-1c4d-4e3b-83b3-cee99ddb0841 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:23.331773 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:23.344052 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-8f967df2-1c4d-4e3b-83b3-cee99ddb0841 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:23.344052 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:23.344052 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:23.344052 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:23.344052 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:23.344052 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:23.344052 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:23.344052 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:23.344052 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:23.344052 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:23.344052 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:23.344052 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:23.344052 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 05:06:23.344052 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 05:06:23.344052 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 05:06:23.345169 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 05:06:23.345169 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:23.345169 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:23.345169 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:23.345169 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:23.345169 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:23.345169 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:23.345169 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:23.345169 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:23.345169 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:23.345169 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:23.345169 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:23.345169 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:23.345169 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:23.345169 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:23.345169 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:23.345169 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:23.346157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 05:06:23.346157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 05:06:23.346157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 05:06:23.346157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 05:06:23.346157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 05:06:23.346157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 05:06:23.346157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:23.346157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:23.346157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 339/657] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:06:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:23.346664 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f967df2-1c4d-4e3b-83b3-cee99ddb0841 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:23.374913 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9e301f-7b9b-4f09-bc23-2387f65c87dd None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:23.397048 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:23.409334 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27c918-029c-4e5f-b5cd-582bfcbc9ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:23.409957 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27c918-029c-4e5f-b5cd-582bfcbc9ddc None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:23.410244 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27c918-029c-4e5f-b5cd-582bfcbc9ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:23.410524 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27c918-029c-4e5f-b5cd-582bfcbc9ddc None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:23.411208 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc27c918-029c-4e5f-b5cd-582bfcbc9ddc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:23.411510 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc27c918-029c-4e5f-b5cd-582bfcbc9ddc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:23.412955 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc27c918-029c-4e5f-b5cd-582bfcbc9ddc None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:23.424769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 319/658] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 05:06:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:23.425605 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27c918-029c-4e5f-b5cd-582bfcbc9ddc None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:23.455318 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f967df2-1c4d-4e3b-83b3-cee99ddb0841 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:23.489078 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b3e2e2-55a8-4322-b3f1-b84608443aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:23.489636 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3e2e2-55a8-4322-b3f1-b84608443aa8 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:23.489911 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3e2e2-55a8-4322-b3f1-b84608443aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:23.490202 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3e2e2-55a8-4322-b3f1-b84608443aa8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:23.497857 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b3e2e2-55a8-4322-b3f1-b84608443aa8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:23.499384 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b3e2e2-55a8-4322-b3f1-b84608443aa8 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:23.510164 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-51b3e2e2-55a8-4322-b3f1-b84608443aa8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 05:06:23.510164 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:23.510164 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:23.510164 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:23.510164 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:23.510164 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:23.510164 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:23.510164 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:23.510164 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:23.510164 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:23.510164 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:23.510164 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:23.510164 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 05:06:23.510164 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 05:06:23.510164 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 05:06:23.510164 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 05:06:23.511283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:23.511283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:23.511283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 05:06:23.511283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:23.511283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 05:06:23.511283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:23.511283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:23.511283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:23.511283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:23.511283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:23.511283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:23.511283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:23.511283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:23.511283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:23.511283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:23.511283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:23.511283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:23.511283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:23.512448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:23.512448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:23.512448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 05:06:23.512448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 05:06:23.512448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 05:06:23.512448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 05:06:23.512448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 05:06:23.512448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 05:06:23.512448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 05:06:23.512448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:23.515842 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 340/659] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:06:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:23.516665 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b3e2e2-55a8-4322-b3f1-b84608443aa8 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:23.539442 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27c918-029c-4e5f-b5cd-582bfcbc9ddc None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:23.574505 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f5164-dc4b-4d27-8925-29fef15f8576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:23.575139 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f5164-dc4b-4d27-8925-29fef15f8576 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:23.575393 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f5164-dc4b-4d27-8925-29fef15f8576 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:23.575635 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f5164-dc4b-4d27-8925-29fef15f8576 None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:23.576472 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570f5164-dc4b-4d27-8925-29fef15f8576 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:23.577018 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570f5164-dc4b-4d27-8925-29fef15f8576 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:23.579467 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570f5164-dc4b-4d27-8925-29fef15f8576 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:23.580570 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:23.601242 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 320/660] 10.222.0.36 () {56 vars in 1139 bytes} [Wed Aug 31 05:06:23 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:23.606211 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f5164-dc4b-4d27-8925-29fef15f8576 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:23.627727 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b3e2e2-55a8-4322-b3f1-b84608443aa8 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:23.655335 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:23.667227 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b4109e-1ad5-4d40-85d2-f1847274160a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:23.667993 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b4109e-1ad5-4d40-85d2-f1847274160a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:23.668372 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b4109e-1ad5-4d40-85d2-f1847274160a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:23.668747 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b4109e-1ad5-4d40-85d2-f1847274160a None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:23.669514 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b4109e-1ad5-4d40-85d2-f1847274160a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:23.669967 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b4109e-1ad5-4d40-85d2-f1847274160a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:23.671488 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b4109e-1ad5-4d40-85d2-f1847274160a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:23.695087 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 341/661] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 05:06:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:23.695087 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b4109e-1ad5-4d40-85d2-f1847274160a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:23.730968 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f5164-dc4b-4d27-8925-29fef15f8576 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:23.766725 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afda1e-dbc0-4378-8156-4fc05c5dfeaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:23.767491 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afda1e-dbc0-4378-8156-4fc05c5dfeaf None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:23.767864 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afda1e-dbc0-4378-8156-4fc05c5dfeaf None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:23.768195 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afda1e-dbc0-4378-8156-4fc05c5dfeaf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:23.768978 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15afda1e-dbc0-4378-8156-4fc05c5dfeaf None admin] RBAC: Adding query filter params (role.id=207bf98e31774cc0b21249de6d96155b, scope.project.id=e01ce1b1282e44a39d659b87f120995e, user.id=c08e6024b0c8420aa274653c2a916edd) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:23.769337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15afda1e-dbc0-4378-8156-4fc05c5dfeaf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:23.770946 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15afda1e-dbc0-4378-8156-4fc05c5dfeaf None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:23.777915 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 321/662] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:06:23 2022] GET /identity/v3/role_assignments?user.id=c08e6024b0c8420aa274653c2a916edd&scope.project.id=e01ce1b1282e44a39d659b87f120995e&role.id=207bf98e31774cc0b21249de6d96155b => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:23.778867 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afda1e-dbc0-4378-8156-4fc05c5dfeaf None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:23.817152 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b4109e-1ad5-4d40-85d2-f1847274160a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:23.846629 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:23.874489 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a341b6-177a-482d-97ea-97a72bd86204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:23.875269 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a341b6-177a-482d-97ea-97a72bd86204 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:23.876112 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a341b6-177a-482d-97ea-97a72bd86204 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:23.876480 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a341b6-177a-482d-97ea-97a72bd86204 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:23.877914 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a341b6-177a-482d-97ea-97a72bd86204 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:23.878342 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a341b6-177a-482d-97ea-97a72bd86204 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:23.880220 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a341b6-177a-482d-97ea-97a72bd86204 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:23.904650 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afda1e-dbc0-4378-8156-4fc05c5dfeaf None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:23.906423 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 342/663] 10.222.0.36 () {56 vars in 1137 bytes} [Wed Aug 31 05:06:23 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:23.907939 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a341b6-177a-482d-97ea-97a72bd86204 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:23.909536 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:23.953239 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b29d49-328a-443c-8565-897a7056cb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:23.953854 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b29d49-328a-443c-8565-897a7056cb63 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:23.954167 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b29d49-328a-443c-8565-897a7056cb63 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:23.954441 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b29d49-328a-443c-8565-897a7056cb63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:24.023663 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a341b6-177a-482d-97ea-97a72bd86204 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:24.028997 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b29d49-328a-443c-8565-897a7056cb63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:24.030593 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b29d49-328a-443c-8565-897a7056cb63 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:24.061680 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d831893-9204-488f-b461-19101d4476a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:24.062423 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d831893-9204-488f-b461-19101d4476a3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:24.062695 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d831893-9204-488f-b461-19101d4476a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:24.063065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d831893-9204-488f-b461-19101d4476a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:24.063793 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d831893-9204-488f-b461-19101d4476a3 None admin] RBAC: Adding query filter params (role.id=207bf98e31774cc0b21249de6d96155b, scope.project.id=e01ce1b1282e44a39d659b87f120995e, user.id=f27665b1574a4db7b7a9771f7d775826) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:24.064086 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d831893-9204-488f-b461-19101d4476a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:24.065508 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d831893-9204-488f-b461-19101d4476a3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:24.071901 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 343/664] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:06:23 2022] GET /identity/v3/role_assignments?user.id=f27665b1574a4db7b7a9771f7d775826&scope.project.id=e01ce1b1282e44a39d659b87f120995e&role.id=207bf98e31774cc0b21249de6d96155b => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:24.073089 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d831893-9204-488f-b461-19101d4476a3 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:24.155692 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 322/665] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:06:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 05:06:24.165816 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b29d49-328a-443c-8565-897a7056cb63 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:24.173414 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:24.175991 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:24.265254 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d831893-9204-488f-b461-19101d4476a3 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:24.303659 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf14f3b3-cf5d-48bd-bedb-57e9abede437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:24.304190 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf14f3b3-cf5d-48bd-bedb-57e9abede437 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:24.304515 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf14f3b3-cf5d-48bd-bedb-57e9abede437 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:24.304771 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b29d49-328a-443c-8565-897a7056cb63 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:24.304885 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf14f3b3-cf5d-48bd-bedb-57e9abede437 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:24.350901 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acedec6-5ab1-4f63-acfd-976a484e7044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:24.351592 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acedec6-5ab1-4f63-acfd-976a484e7044 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:24.351880 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acedec6-5ab1-4f63-acfd-976a484e7044 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:24.352151 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acedec6-5ab1-4f63-acfd-976a484e7044 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:24.364791 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acedec6-5ab1-4f63-acfd-976a484e7044 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:24.366344 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acedec6-5ab1-4f63-acfd-976a484e7044 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:24.382150 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-8acedec6-5ab1-4f63-acfd-976a484e7044 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:24.382150 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:24.382150 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:24.382150 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:24.382150 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:24.382150 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:24.382150 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:24.382150 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:24.382150 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:24.382150 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:24.382150 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:24.382150 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:24.382150 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:24.382150 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:24.382150 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:24.382150 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:24.383213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:24.383213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:24.383213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:24.383213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:24.383213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:24.383213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:24.383213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:24.383213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:24.383213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:24.383213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:24.383213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:24.383213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:24.383213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:24.383213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:24.383213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:24.383213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:24.383213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:24.384173 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:24.384173 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:24.384173 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:24.384173 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 323/666] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:24.389103 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acedec6-5ab1-4f63-acfd-976a484e7044 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:24.398272 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf14f3b3-cf5d-48bd-bedb-57e9abede437 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:24.400542 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf14f3b3-cf5d-48bd-bedb-57e9abede437 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:24.447051 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:24.455012 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:24.511631 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acedec6-5ab1-4f63-acfd-976a484e7044 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:24.527601 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 344/667] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:06:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 05:06:24.532659 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf14f3b3-cf5d-48bd-bedb-57e9abede437 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:24.549549 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eafb060-f2a1-415c-b8fe-7f82f68e7ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:24.550592 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eafb060-f2a1-415c-b8fe-7f82f68e7ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:24.550952 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eafb060-f2a1-415c-b8fe-7f82f68e7ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:24.551287 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eafb060-f2a1-415c-b8fe-7f82f68e7ac5 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:24.552209 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eafb060-f2a1-415c-b8fe-7f82f68e7ac5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:24.552572 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eafb060-f2a1-415c-b8fe-7f82f68e7ac5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:24.554711 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eafb060-f2a1-415c-b8fe-7f82f68e7ac5 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:24.571252 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 324/668] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:06:24.577758 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eafb060-f2a1-415c-b8fe-7f82f68e7ac5 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:24.680727 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf14f3b3-cf5d-48bd-bedb-57e9abede437 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:24.718048 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90f0c63-567c-4c50-a35b-f7e68bc6869b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:24.718687 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f0c63-567c-4c50-a35b-f7e68bc6869b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:24.718963 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f0c63-567c-4c50-a35b-f7e68bc6869b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:24.719273 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f0c63-567c-4c50-a35b-f7e68bc6869b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:24.729956 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90f0c63-567c-4c50-a35b-f7e68bc6869b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:24.730127 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eafb060-f2a1-415c-b8fe-7f82f68e7ac5 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:24.731433 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90f0c63-567c-4c50-a35b-f7e68bc6869b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:24.733282 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:24.748169 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:24.749661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-d90f0c63-567c-4c50-a35b-f7e68bc6869b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:24.749661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:24.749661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:24.749661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:24.749661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:24.749661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:24.749661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:24.749661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:24.749661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:24.749661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:24.749661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:24.749661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:24.749661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:24.749661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:24.749661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:24.749661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:24.750827 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:24.750827 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:24.750827 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:24.750827 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:24.750827 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:24.750827 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:24.750827 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:24.750827 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:24.750827 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:24.750827 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:24.750827 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:24.750827 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:24.750827 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:24.750827 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:24.750827 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:24.750827 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:24.750827 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:24.752006 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:24.752006 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:24.752006 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:24.753921 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 345/669] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:06:24.755560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812bbad6-8e2a-4ae1-a663-81374942d392 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:24.756024 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812bbad6-8e2a-4ae1-a663-81374942d392 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:24.756571 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812bbad6-8e2a-4ae1-a663-81374942d392 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:24.761891 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 346/670] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:24.762789 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812bbad6-8e2a-4ae1-a663-81374942d392 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:24.767468 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee3848-4fc6-4346-bc45-2ca0579ac342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:24.768349 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee3848-4fc6-4346-bc45-2ca0579ac342 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:24.768837 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee3848-4fc6-4346-bc45-2ca0579ac342 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:24.769310 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee3848-4fc6-4346-bc45-2ca0579ac342 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:24.776807 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee3848-4fc6-4346-bc45-2ca0579ac342 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:24.779223 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee3848-4fc6-4346-bc45-2ca0579ac342 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:24.795249 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-82ee3848-4fc6-4346-bc45-2ca0579ac342 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:24.795249 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:24.795249 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:24.795249 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:24.795249 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:24.795249 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:24.795249 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:24.795249 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:24.795249 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:24.795249 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:24.795249 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:24.795249 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:24.795249 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 05:06:24.795249 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 05:06:24.795249 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 05:06:24.796301 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 05:06:24.796301 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:24.796301 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:24.796301 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:24.796301 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:24.796301 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:24.796301 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:24.796301 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:24.796301 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:24.796301 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:24.796301 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:24.796301 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:24.796301 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:24.796301 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:24.796301 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:24.796301 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:24.796301 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:24.797199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 05:06:24.797199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 05:06:24.797199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 05:06:24.797199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 05:06:24.797199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 05:06:24.797199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 05:06:24.797199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:24.797199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:24.800663 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 325/671] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:06:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:06:24.802501 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d8162-7d9c-460f-8839-2d865e2accc3 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:24.802998 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d8162-7d9c-460f-8839-2d865e2accc3 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:24.803481 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d8162-7d9c-460f-8839-2d865e2accc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:24.822527 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-391d8162-7d9c-460f-8839-2d865e2accc3 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:24.824119 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-391d8162-7d9c-460f-8839-2d865e2accc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:24.883330 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812bbad6-8e2a-4ae1-a663-81374942d392 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:24.904119 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 326/672] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:06:24.905173 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d8162-7d9c-460f-8839-2d865e2accc3 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:24.935158 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b7f869-7139-40d0-af12-4ba14583d3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:24.936029 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7f869-7139-40d0-af12-4ba14583d3a2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:24.936359 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7f869-7139-40d0-af12-4ba14583d3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:24.936714 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7f869-7139-40d0-af12-4ba14583d3a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:24.937361 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b7f869-7139-40d0-af12-4ba14583d3a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:24.937701 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b7f869-7139-40d0-af12-4ba14583d3a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:24.939150 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b7f869-7139-40d0-af12-4ba14583d3a2 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:24.965675 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 347/673] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:24.970985 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b7f869-7139-40d0-af12-4ba14583d3a2 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:25.011062 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d8162-7d9c-460f-8839-2d865e2accc3 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:25.019948 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:25.049732 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7d0cd-65c2-4c2f-a993-aebafcaa5453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:25.050679 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7d0cd-65c2-4c2f-a993-aebafcaa5453 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:25.051136 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7d0cd-65c2-4c2f-a993-aebafcaa5453 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:25.051623 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7d0cd-65c2-4c2f-a993-aebafcaa5453 None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:25.053038 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7d0cd-65c2-4c2f-a993-aebafcaa5453 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:25.053931 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7d0cd-65c2-4c2f-a993-aebafcaa5453 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:25.055771 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7d0cd-65c2-4c2f-a993-aebafcaa5453 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:25.069870 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 327/674] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 05:06:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:25.079625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7d0cd-65c2-4c2f-a993-aebafcaa5453 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:25.080360 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:25.138139 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b7f869-7139-40d0-af12-4ba14583d3a2 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:25.205747 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b635526-a31d-4b4a-b478-e666635fe2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:25.206403 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b635526-a31d-4b4a-b478-e666635fe2ce None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:25.206675 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b635526-a31d-4b4a-b478-e666635fe2ce None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:25.206945 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b635526-a31d-4b4a-b478-e666635fe2ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:25.214875 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b635526-a31d-4b4a-b478-e666635fe2ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:25.216569 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b635526-a31d-4b4a-b478-e666635fe2ce None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:25.239034 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-7b635526-a31d-4b4a-b478-e666635fe2ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:25.239034 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:25.239034 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:25.239034 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:25.239034 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:25.239034 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:25.239034 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:25.239034 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:25.239034 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:25.239034 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:25.239034 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:25.239034 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:25.239034 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 05:06:25.239034 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 05:06:25.239034 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 05:06:25.239977 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 05:06:25.239977 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:25.239977 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:25.239977 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:25.239977 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:25.239977 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:25.239977 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:25.239977 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:25.239977 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:25.239977 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:25.239977 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:25.239977 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:25.239977 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:25.239977 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:25.239977 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:25.239977 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:25.239977 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:25.240893 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 05:06:25.240893 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 05:06:25.240893 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 05:06:25.240893 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 05:06:25.240893 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 05:06:25.240893 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 05:06:25.240893 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:25.240893 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:25.243619 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 348/675] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:06:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:06:25.249308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b635526-a31d-4b4a-b478-e666635fe2ce None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:25.257384 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7d0cd-65c2-4c2f-a993-aebafcaa5453 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:25.291320 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f505142-26b7-4c50-9b6e-86feb4da5a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:25.291920 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f505142-26b7-4c50-9b6e-86feb4da5a70 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:25.292195 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f505142-26b7-4c50-9b6e-86feb4da5a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:25.292461 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f505142-26b7-4c50-9b6e-86feb4da5a70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:25.293181 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f505142-26b7-4c50-9b6e-86feb4da5a70 None admin] RBAC: Adding query filter params (role.id=207bf98e31774cc0b21249de6d96155b, scope.project.id=e01ce1b1282e44a39d659b87f120995e, user.id=090e1409506542f6831e1f65d44aae74) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:25.293487 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f505142-26b7-4c50-9b6e-86feb4da5a70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:25.294999 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f505142-26b7-4c50-9b6e-86feb4da5a70 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:25.306027 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 328/676] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:06:25 2022] GET /identity/v3/role_assignments?user.id=090e1409506542f6831e1f65d44aae74&scope.project.id=e01ce1b1282e44a39d659b87f120995e&role.id=207bf98e31774cc0b21249de6d96155b => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:25.306027 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f505142-26b7-4c50-9b6e-86feb4da5a70 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:25.338954 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:25.390250 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b635526-a31d-4b4a-b478-e666635fe2ce None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:25.408889 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:25.416533 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f505142-26b7-4c50-9b6e-86feb4da5a70 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:25.437883 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ac6995-8d8b-4bb8-810d-3aa6523d4197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:25.438803 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac6995-8d8b-4bb8-810d-3aa6523d4197 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:25.439227 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac6995-8d8b-4bb8-810d-3aa6523d4197 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:25.440202 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac6995-8d8b-4bb8-810d-3aa6523d4197 None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:25.441058 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ac6995-8d8b-4bb8-810d-3aa6523d4197 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:25.441756 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ac6995-8d8b-4bb8-810d-3aa6523d4197 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:25.444036 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ac6995-8d8b-4bb8-810d-3aa6523d4197 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:25.456179 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ca3da4-e1f6-4a89-89e1-dac6ab367b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:25.456971 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca3da4-e1f6-4a89-89e1-dac6ab367b84 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:25.457380 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca3da4-e1f6-4a89-89e1-dac6ab367b84 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:25.458076 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca3da4-e1f6-4a89-89e1-dac6ab367b84 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:25.462390 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 349/677] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 05:06:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:06:25.468682 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ac6995-8d8b-4bb8-810d-3aa6523d4197 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:25.478793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ca3da4-e1f6-4a89-89e1-dac6ab367b84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:06:25.485839 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-13ca3da4-e1f6-4a89-89e1-dac6ab367b84 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 05:06:25.485839 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:25.485839 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:25.485839 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:25.485839 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:25.485839 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:25.485839 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:25.485839 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:25.485839 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:25.485839 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:25.485839 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:25.485839 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:25.485839 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:06:25.485839 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:06:25.485839 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:06:25.485839 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:06:25.487037 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:06:25.487037 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:06:25.487037 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:06:25.487037 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:06:25.487037 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:25.487037 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:25.487037 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:25.487037 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:25.487037 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:25.487037 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:25.487037 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:25.487037 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:25.487037 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:25.487037 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:25.487037 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:25.487037 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:25.488114 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:25.488114 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:25.488114 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:25.488114 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:25.488114 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:06:25.488114 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:06:25.488114 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:06:25.488114 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:06:25.488114 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:06:25.488114 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:06:25.488114 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 05:06:25.488114 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:25.490892 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 329/678] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 05:06:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:25.500095 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ca3da4-e1f6-4a89-89e1-dac6ab367b84 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:25.585976 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ac6995-8d8b-4bb8-810d-3aa6523d4197 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:25.633952 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e477ea78-b3c7-462f-9ebc-019b531f9d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:25.634681 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477ea78-b3c7-462f-9ebc-019b531f9d56 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:25.635085 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477ea78-b3c7-462f-9ebc-019b531f9d56 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:25.635455 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477ea78-b3c7-462f-9ebc-019b531f9d56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:25.636282 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e477ea78-b3c7-462f-9ebc-019b531f9d56 None admin] RBAC: Adding query filter params (role.id=207bf98e31774cc0b21249de6d96155b, scope.project.id=e01ce1b1282e44a39d659b87f120995e, user.id=a8300075371341a6b9cd4dcf8ec7fa8f) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:25.636706 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e477ea78-b3c7-462f-9ebc-019b531f9d56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:25.638339 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e477ea78-b3c7-462f-9ebc-019b531f9d56 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:25.649924 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:25.650716 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 350/679] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:06:25 2022] GET /identity/v3/role_assignments?user.id=a8300075371341a6b9cd4dcf8ec7fa8f&scope.project.id=e01ce1b1282e44a39d659b87f120995e&role.id=207bf98e31774cc0b21249de6d96155b => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:06:25.664728 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ca3da4-e1f6-4a89-89e1-dac6ab367b84 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:25.703760 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb6350e-2ef4-465f-a9e9-88946f63370d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:25.704546 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6350e-2ef4-465f-a9e9-88946f63370d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:25.704921 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6350e-2ef4-465f-a9e9-88946f63370d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:25.705307 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6350e-2ef4-465f-a9e9-88946f63370d None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:25.706146 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb6350e-2ef4-465f-a9e9-88946f63370d None admin] RBAC: Adding query filter params (name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:25.706546 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb6350e-2ef4-465f-a9e9-88946f63370d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:25.708083 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb6350e-2ef4-465f-a9e9-88946f63370d None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:25.718438 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 330/680] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Aug 31 05:06:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:25.723956 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e477ea78-b3c7-462f-9ebc-019b531f9d56 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:25.774354 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:25.848254 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e477ea78-b3c7-462f-9ebc-019b531f9d56 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:25.889947 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4557466-6b5c-463d-9fcd-802e6a4df353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:25.890680 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4557466-6b5c-463d-9fcd-802e6a4df353 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:25.891066 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4557466-6b5c-463d-9fcd-802e6a4df353 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:25.891420 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4557466-6b5c-463d-9fcd-802e6a4df353 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:25.903289 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4557466-6b5c-463d-9fcd-802e6a4df353 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:25.905331 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4557466-6b5c-463d-9fcd-802e6a4df353 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:25.920420 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-e4557466-6b5c-463d-9fcd-802e6a4df353 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:25.920420 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:25.920420 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:25.920420 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:25.920420 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:25.920420 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:25.920420 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:25.920420 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:25.920420 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:25.920420 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:25.920420 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:25.920420 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:25.920420 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:25.920420 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:25.920420 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:25.920420 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:25.921710 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:25.921710 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:25.921710 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:25.921710 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:25.921710 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:25.921710 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:25.921710 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:25.921710 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:25.921710 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:25.921710 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:25.921710 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:25.921710 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:25.921710 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:25.921710 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:25.921710 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:25.921710 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:25.921710 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:25.922880 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:25.922880 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:25.922880 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:25.924077 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 351/681] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:25.930637 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb6350e-2ef4-465f-a9e9-88946f63370d None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:26.034537 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb6350e-2ef4-465f-a9e9-88946f63370d None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:26.069150 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8019160f-a4b0-4aae-abba-923c9ecb2647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:26.069760 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019160f-a4b0-4aae-abba-923c9ecb2647 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:26.070042 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a22b5e-6149-4115-b1e2-2e5cab0776d0 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:26.070214 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019160f-a4b0-4aae-abba-923c9ecb2647 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:26.070390 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a22b5e-6149-4115-b1e2-2e5cab0776d0 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:26.070555 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019160f-a4b0-4aae-abba-923c9ecb2647 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:26.070699 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a22b5e-6149-4115-b1e2-2e5cab0776d0 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:26.071044 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8019160f-a4b0-4aae-abba-923c9ecb2647 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:26.071344 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8019160f-a4b0-4aae-abba-923c9ecb2647 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:26.072236 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 352/682] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 05:06:26.072710 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8019160f-a4b0-4aae-abba-923c9ecb2647 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:26.077356 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb978d8f-a7a8-4a60-8c42-370b66a77e7f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:26.077633 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb978d8f-a7a8-4a60-8c42-370b66a77e7f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:26.077900 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb978d8f-a7a8-4a60-8c42-370b66a77e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:26.085500 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 331/683] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:26.090863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8019160f-a4b0-4aae-abba-923c9ecb2647 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:26.110691 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:26.115473 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-eb978d8f-a7a8-4a60-8c42-370b66a77e7f None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:26.116678 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-eb978d8f-a7a8-4a60-8c42-370b66a77e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:26.198586 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 353/684] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:06:26.285042 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8019160f-a4b0-4aae-abba-923c9ecb2647 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:26.335906 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a944c-0de2-4193-bc05-cc3da15b4134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:26.336538 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a944c-0de2-4193-bc05-cc3da15b4134 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:26.336816 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a944c-0de2-4193-bc05-cc3da15b4134 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:26.337080 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a944c-0de2-4193-bc05-cc3da15b4134 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:26.349242 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a944c-0de2-4193-bc05-cc3da15b4134 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:26.350734 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a944c-0de2-4193-bc05-cc3da15b4134 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:26.364080 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-735a944c-0de2-4193-bc05-cc3da15b4134 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 05:06:26.364080 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:26.364080 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:26.364080 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:26.364080 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:26.364080 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:26.364080 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:26.364080 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:26.364080 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:26.364080 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:26.364080 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:26.364080 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:26.364080 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 05:06:26.364080 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 05:06:26.364080 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 05:06:26.364080 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 05:06:26.365053 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:26.365053 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:26.365053 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 05:06:26.365053 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:26.365053 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 05:06:26.365053 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:26.365053 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:26.365053 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:26.365053 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:26.365053 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:26.365053 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:26.365053 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:26.365053 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:26.365053 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:26.365053 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:26.365053 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:26.365053 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:26.365053 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:26.366074 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:26.366074 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:26.366074 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 05:06:26.366074 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 05:06:26.366074 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 05:06:26.366074 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 05:06:26.366074 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 05:06:26.366074 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 05:06:26.366074 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 05:06:26.366074 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:26.366074 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 332/685] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 05:06:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:06:26.371574 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb978d8f-a7a8-4a60-8c42-370b66a77e7f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:26.410889 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:26.497663 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb978d8f-a7a8-4a60-8c42-370b66a77e7f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:26.531260 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780fa53-8003-4708-a049-bcd1ade45bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:26.531928 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780fa53-8003-4708-a049-bcd1ade45bc0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:26.532202 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780fa53-8003-4708-a049-bcd1ade45bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:26.532466 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780fa53-8003-4708-a049-bcd1ade45bc0 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:26.533187 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6780fa53-8003-4708-a049-bcd1ade45bc0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:26.533489 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6780fa53-8003-4708-a049-bcd1ade45bc0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:26.534944 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6780fa53-8003-4708-a049-bcd1ade45bc0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:26.561151 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 354/686] 10.222.0.36 () {56 vars in 1135 bytes} [Wed Aug 31 05:06:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:26.566923 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a944c-0de2-4193-bc05-cc3da15b4134 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:26.621581 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780fa53-8003-4708-a049-bcd1ade45bc0 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:26.645095 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:26.676446 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a944c-0de2-4193-bc05-cc3da15b4134 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:26.710584 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bed342f-37f9-4924-bd0b-4e02d29c293a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:26.711195 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed342f-37f9-4924-bd0b-4e02d29c293a None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:26.711471 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed342f-37f9-4924-bd0b-4e02d29c293a None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:26.711739 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed342f-37f9-4924-bd0b-4e02d29c293a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:26.763567 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:26.781420 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bed342f-37f9-4924-bd0b-4e02d29c293a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:26.782946 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bed342f-37f9-4924-bd0b-4e02d29c293a None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:26.786817 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780fa53-8003-4708-a049-bcd1ade45bc0 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:26.822848 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a2806c-97af-45c6-a04c-020808c58940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:26.823529 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2806c-97af-45c6-a04c-020808c58940 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:26.823808 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2806c-97af-45c6-a04c-020808c58940 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:26.824073 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2806c-97af-45c6-a04c-020808c58940 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:26.835064 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a2806c-97af-45c6-a04c-020808c58940 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:26.836612 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a2806c-97af-45c6-a04c-020808c58940 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:26.852199 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-01a2806c-97af-45c6-a04c-020808c58940 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:26.852199 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:26.852199 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:26.852199 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:26.852199 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:26.852199 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:26.852199 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:26.852199 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:26.852199 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:26.852199 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:26.852199 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:26.852199 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:26.852199 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:26.852199 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:26.852199 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:26.852199 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:26.853154 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:26.853154 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:26.853154 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:26.853154 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:26.853154 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:26.853154 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:26.853154 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:26.853154 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:26.853154 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:26.853154 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:26.853154 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:26.853154 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:26.853154 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:26.853154 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:26.853154 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:26.853154 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:26.853154 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:26.854087 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:26.854087 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:26.854087 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:26.854275 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:26.854898 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 355/687] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:06:26.856640 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c2a16f-1d81-4671-8fbe-762425bf9b42 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:26.857028 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c2a16f-1d81-4671-8fbe-762425bf9b42 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:26.857453 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c2a16f-1d81-4671-8fbe-762425bf9b42 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:26.863928 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 356/688] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:26.864880 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c2a16f-1d81-4671-8fbe-762425bf9b42 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:26.891323 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 333/689] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:06:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 05:06:26.892871 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713d186-1c20-49a8-817f-28cf81f7710e None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:26.893158 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713d186-1c20-49a8-817f-28cf81f7710e None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:26.893428 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713d186-1c20-49a8-817f-28cf81f7710e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:26.912820 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-6713d186-1c20-49a8-817f-28cf81f7710e None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:26.914017 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-6713d186-1c20-49a8-817f-28cf81f7710e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:26.976770 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c2a16f-1d81-4671-8fbe-762425bf9b42 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:26.984467 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 334/690] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:06:26.985472 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6713d186-1c20-49a8-817f-28cf81f7710e None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:27.026976 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:27.042503 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f87a8d8-affb-418e-8856-3f45c54f2197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:27.043358 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87a8d8-affb-418e-8856-3f45c54f2197 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:27.043777 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87a8d8-affb-418e-8856-3f45c54f2197 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:27.044193 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87a8d8-affb-418e-8856-3f45c54f2197 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:27.045022 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f87a8d8-affb-418e-8856-3f45c54f2197 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:27.045453 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f87a8d8-affb-418e-8856-3f45c54f2197 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:27.046986 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f87a8d8-affb-418e-8856-3f45c54f2197 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:27.074864 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 357/691] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:27.080244 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f87a8d8-affb-418e-8856-3f45c54f2197 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:27.163560 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6713d186-1c20-49a8-817f-28cf81f7710e None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:27.167768 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:27.225345 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ab94b-bda5-43c0-95ec-a9b3fa2f9588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:27.226461 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ab94b-bda5-43c0-95ec-a9b3fa2f9588 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:27.227732 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ab94b-bda5-43c0-95ec-a9b3fa2f9588 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:27.228851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ab94b-bda5-43c0-95ec-a9b3fa2f9588 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:27.251040 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1ab94b-bda5-43c0-95ec-a9b3fa2f9588 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:27.253585 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1ab94b-bda5-43c0-95ec-a9b3fa2f9588 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:27.256895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f87a8d8-affb-418e-8856-3f45c54f2197 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:27.275203 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-2f1ab94b-bda5-43c0-95ec-a9b3fa2f9588 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:27.275203 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:27.275203 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:27.275203 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:27.275203 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:27.275203 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:27.275203 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:27.275203 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:27.275203 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:27.275203 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:27.275203 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:27.275203 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:27.275203 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:27.275203 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:27.275203 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:27.275203 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:27.276394 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:27.276394 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:27.276394 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:27.276394 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:27.276394 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:27.276394 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:27.276394 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:27.276394 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:27.276394 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:27.276394 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:27.276394 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:27.276394 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:27.276394 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:27.276394 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:27.276394 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:27.276394 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:27.276394 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:27.277606 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:27.277606 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:27.277606 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:27.280755 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 335/692] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:27.286554 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ab94b-bda5-43c0-95ec-a9b3fa2f9588 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:27.336268 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102336ad-6860-4616-af85-a58a77860798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:27.337542 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102336ad-6860-4616-af85-a58a77860798 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:27.337957 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102336ad-6860-4616-af85-a58a77860798 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:27.338674 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102336ad-6860-4616-af85-a58a77860798 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:27.339520 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102336ad-6860-4616-af85-a58a77860798 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:27.341384 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102336ad-6860-4616-af85-a58a77860798 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:27.369475 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 358/693] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Aug 31 05:06:27 2022] POST /identity/v3/users => generated 254 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:06:27.371477 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102336ad-6860-4616-af85-a58a77860798 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:27.415665 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ab94b-bda5-43c0-95ec-a9b3fa2f9588 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:27.429457 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:27.463957 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5663de-d69a-44ec-b57c-570e9e6fe2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:27.464590 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5663de-d69a-44ec-b57c-570e9e6fe2b6 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:27.464854 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5663de-d69a-44ec-b57c-570e9e6fe2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:27.465123 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5663de-d69a-44ec-b57c-570e9e6fe2b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:27.465842 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5663de-d69a-44ec-b57c-570e9e6fe2b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:27.466176 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5663de-d69a-44ec-b57c-570e9e6fe2b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:27.467677 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5663de-d69a-44ec-b57c-570e9e6fe2b6 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:27.485266 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 336/694] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:06:27.490751 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5663de-d69a-44ec-b57c-570e9e6fe2b6 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:27.503121 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102336ad-6860-4616-af85-a58a77860798 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:27.527790 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:27.542937 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c85da7-ecb0-4283-827f-b1603b30dc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:27.543870 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c85da7-ecb0-4283-827f-b1603b30dc67 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:27.544294 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c85da7-ecb0-4283-827f-b1603b30dc67 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:27.544779 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c85da7-ecb0-4283-827f-b1603b30dc67 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:27.557273 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c85da7-ecb0-4283-827f-b1603b30dc67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:06:27.563163 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-c8c85da7-ecb0-4283-827f-b1603b30dc67 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:06:27.563163 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:27.563163 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:27.563163 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:27.563163 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:27.563163 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:27.563163 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:27.563163 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:27.563163 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:27.563163 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:27.563163 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:27.563163 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:27.563163 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:06:27.563163 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:06:27.563163 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:06:27.563163 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:06:27.564316 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:06:27.564316 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:06:27.564316 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:06:27.564316 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:06:27.564316 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:27.564316 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:27.564316 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:27.564316 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:27.564316 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:27.564316 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:27.564316 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:27.564316 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:27.564316 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:27.564316 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:27.564316 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:27.564316 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:27.565303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:27.565303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:27.565303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:27.565303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:27.565303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:06:27.565303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:06:27.565303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:06:27.565303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:06:27.565303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:06:27.565303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:06:27.565303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:06:27.565303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:27.567695 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 359/695] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 05:06:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:27.573844 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c85da7-ecb0-4283-827f-b1603b30dc67 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:27.686103 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c85da7-ecb0-4283-827f-b1603b30dc67 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:27.701688 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:27.715596 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5663de-d69a-44ec-b57c-570e9e6fe2b6 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:27.735817 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3775e049-1492-4b7b-b444-2d0bbba43911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:27.737008 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775e049-1492-4b7b-b444-2d0bbba43911 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:27.737434 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775e049-1492-4b7b-b444-2d0bbba43911 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:27.738130 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775e049-1492-4b7b-b444-2d0bbba43911 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:27.739327 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3775e049-1492-4b7b-b444-2d0bbba43911 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:27.739803 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3775e049-1492-4b7b-b444-2d0bbba43911 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:27.741347 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3775e049-1492-4b7b-b444-2d0bbba43911 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:27.753037 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 360/696] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:06:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:27.758799 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3775e049-1492-4b7b-b444-2d0bbba43911 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:27.765691 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb137456-e2c6-4251-a611-4092a585321e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:27.766637 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb137456-e2c6-4251-a611-4092a585321e None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:27.767092 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb137456-e2c6-4251-a611-4092a585321e None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:27.767537 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb137456-e2c6-4251-a611-4092a585321e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:27.775414 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb137456-e2c6-4251-a611-4092a585321e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:27.779113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb137456-e2c6-4251-a611-4092a585321e None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:27.791778 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-fb137456-e2c6-4251-a611-4092a585321e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:27.791778 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:27.791778 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:27.791778 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:27.791778 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:27.791778 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:27.791778 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:27.791778 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:27.791778 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:27.791778 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:27.791778 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:27.791778 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:27.791778 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 05:06:27.791778 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 05:06:27.791778 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 05:06:27.792711 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 05:06:27.792711 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:27.792711 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:27.792711 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:27.792711 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:27.792711 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:27.792711 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:27.792711 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:27.792711 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:27.792711 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:27.792711 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:27.792711 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:27.792711 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:27.792711 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:27.792711 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:27.792711 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:27.792711 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:27.793625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 05:06:27.793625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 05:06:27.793625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 05:06:27.793625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 05:06:27.793625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 05:06:27.793625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 05:06:27.793625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:27.793625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:27.796311 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 337/697] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:06:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:27.801895 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb137456-e2c6-4251-a611-4092a585321e None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:27.871418 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3775e049-1492-4b7b-b444-2d0bbba43911 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:27.884040 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:27.912797 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1f5fd5-2e2c-467d-9d91-9167dd2fb5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:27.914090 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f5fd5-2e2c-467d-9d91-9167dd2fb5e2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:27.914975 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f5fd5-2e2c-467d-9d91-9167dd2fb5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:27.915666 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f5fd5-2e2c-467d-9d91-9167dd2fb5e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:27.917298 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb137456-e2c6-4251-a611-4092a585321e None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:27.929310 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1f5fd5-2e2c-467d-9d91-9167dd2fb5e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:27.931800 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1f5fd5-2e2c-467d-9d91-9167dd2fb5e2 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:27.952029 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-ae1f5fd5-2e2c-467d-9d91-9167dd2fb5e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:27.952029 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:27.952029 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:27.952029 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:27.952029 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:27.952029 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:27.952029 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:27.952029 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:27.952029 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:27.952029 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:27.952029 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:27.952029 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:27.952029 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:27.952029 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:27.952029 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:27.952029 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:27.953063 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:27.953063 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:27.953063 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:27.953063 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:27.953063 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:27.953063 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:27.953063 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:27.953063 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:27.953063 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:27.953063 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:27.953063 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:27.953063 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:27.953063 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:27.953063 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:27.953063 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:27.953063 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:27.953063 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:27.953956 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:27.953956 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:27.953956 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:27.955838 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f8061-fbdc-4137-bffe-ea2c03708745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:27.956301 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 361/698] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:06:27.956402 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f8061-fbdc-4137-bffe-ea2c03708745 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:27.956615 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f8061-fbdc-4137-bffe-ea2c03708745 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:27.956890 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f8061-fbdc-4137-bffe-ea2c03708745 None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:27.958249 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347427fb-a345-46b1-8ee6-e39d76973e73 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:27.958717 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347427fb-a345-46b1-8ee6-e39d76973e73 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:27.959170 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347427fb-a345-46b1-8ee6-e39d76973e73 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:27.960821 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 362/699] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:27.964897 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f8061-fbdc-4137-bffe-ea2c03708745 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:27.964897 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f8061-fbdc-4137-bffe-ea2c03708745 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:27.964897 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f8061-fbdc-4137-bffe-ea2c03708745 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:27.966534 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347427fb-a345-46b1-8ee6-e39d76973e73 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:27.979614 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 338/700] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 05:06:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:27.981322 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f520b-a8f6-47ae-8fd8-8c774f29a022 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:27.981611 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f520b-a8f6-47ae-8fd8-8c774f29a022 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:27.981885 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f520b-a8f6-47ae-8fd8-8c774f29a022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:27.994303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:28.006740 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-901f520b-a8f6-47ae-8fd8-8c774f29a022 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:28.008312 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-901f520b-a8f6-47ae-8fd8-8c774f29a022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:28.087960 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 339/701] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:06:28.088750 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901f520b-a8f6-47ae-8fd8-8c774f29a022 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:28.093149 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347427fb-a345-46b1-8ee6-e39d76973e73 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:28.131725 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6050d3-9aed-4805-bda5-67eefeabdbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:28.132368 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6050d3-9aed-4805-bda5-67eefeabdbc3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:28.132692 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6050d3-9aed-4805-bda5-67eefeabdbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:28.132970 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6050d3-9aed-4805-bda5-67eefeabdbc3 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:28.133635 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6050d3-9aed-4805-bda5-67eefeabdbc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:28.133938 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6050d3-9aed-4805-bda5-67eefeabdbc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:28.135375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6050d3-9aed-4805-bda5-67eefeabdbc3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:28.148236 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 363/702] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:28.149459 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91774f27-3962-436a-8d2c-32c4bb47f2c8 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:28.149737 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91774f27-3962-436a-8d2c-32c4bb47f2c8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:28.150033 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91774f27-3962-436a-8d2c-32c4bb47f2c8 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:28.151586 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 364/703] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:28.154524 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91774f27-3962-436a-8d2c-32c4bb47f2c8 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:28.171287 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:28.210467 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901f520b-a8f6-47ae-8fd8-8c774f29a022 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:28.240843 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:28.247247 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f653d-9325-4158-a46f-6726d99540c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:28.247889 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f653d-9325-4158-a46f-6726d99540c6 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:28.248166 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f653d-9325-4158-a46f-6726d99540c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:28.248426 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f653d-9325-4158-a46f-6726d99540c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:28.249157 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f653d-9325-4158-a46f-6726d99540c6 None admin] RBAC: Adding query filter params (role.id=207bf98e31774cc0b21249de6d96155b, scope.project.id=e01ce1b1282e44a39d659b87f120995e, user.id=d9b82080a73d4c4a8ec08b0e67d6d11c) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:28.249459 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f653d-9325-4158-a46f-6726d99540c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:28.251041 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f653d-9325-4158-a46f-6726d99540c6 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:28.262901 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91774f27-3962-436a-8d2c-32c4bb47f2c8 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:28.266592 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 340/704] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:06:28 2022] GET /identity/v3/role_assignments?user.id=d9b82080a73d4c4a8ec08b0e67d6d11c&scope.project.id=e01ce1b1282e44a39d659b87f120995e&role.id=207bf98e31774cc0b21249de6d96155b => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:28.268070 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def306f-5cbe-48d2-b7fa-d3698918939a None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:28.268353 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def306f-5cbe-48d2-b7fa-d3698918939a None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:28.268625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def306f-5cbe-48d2-b7fa-d3698918939a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:28.299664 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-7def306f-5cbe-48d2-b7fa-d3698918939a None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:28.300049 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc1891a-55d3-4020-ac68-eb5b75e95ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:28.300574 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1891a-55d3-4020-ac68-eb5b75e95ef8 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:28.300887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1891a-55d3-4020-ac68-eb5b75e95ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:28.301152 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1891a-55d3-4020-ac68-eb5b75e95ef8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:28.301328 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-7def306f-5cbe-48d2-b7fa-d3698918939a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:28.309138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc1891a-55d3-4020-ac68-eb5b75e95ef8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:28.310741 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc1891a-55d3-4020-ac68-eb5b75e95ef8 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:28.326611 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-2bc1891a-55d3-4020-ac68-eb5b75e95ef8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 05:06:28.326611 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:28.326611 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:28.326611 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:28.326611 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:28.326611 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:28.326611 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:28.326611 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:28.326611 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:28.326611 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:28.326611 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:28.326611 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:28.326611 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 05:06:28.326611 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 05:06:28.326611 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 05:06:28.326611 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 05:06:28.328930 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:28.328930 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:28.328930 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 05:06:28.328930 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:28.328930 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 05:06:28.328930 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:28.328930 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:28.328930 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:28.328930 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:28.328930 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:28.328930 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:28.328930 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:28.328930 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:28.328930 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:28.328930 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:28.328930 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:28.328930 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:28.328930 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:28.330301 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:28.330301 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:28.330301 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 05:06:28.330301 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 05:06:28.330301 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 05:06:28.330301 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 05:06:28.330301 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 05:06:28.330301 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 05:06:28.330301 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 05:06:28.330301 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:28.330301 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 365/705] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Aug 31 05:06:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:06:28.337544 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc1891a-55d3-4020-ac68-eb5b75e95ef8 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:28.419027 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 341/706] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:06:28.420394 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7def306f-5cbe-48d2-b7fa-d3698918939a None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:28.457502 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc1891a-55d3-4020-ac68-eb5b75e95ef8 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:28.484391 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:28.519040 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2acc50-04b8-4d0c-b1ed-04b44c083660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:28.519040 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2acc50-04b8-4d0c-b1ed-04b44c083660 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:28.519040 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2acc50-04b8-4d0c-b1ed-04b44c083660 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:28.519424 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2acc50-04b8-4d0c-b1ed-04b44c083660 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:28.519993 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2acc50-04b8-4d0c-b1ed-04b44c083660 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:28.520306 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2acc50-04b8-4d0c-b1ed-04b44c083660 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:28.521708 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2acc50-04b8-4d0c-b1ed-04b44c083660 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:28.533863 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:28.535909 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7def306f-5cbe-48d2-b7fa-d3698918939a None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:28.547761 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 366/707] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Aug 31 05:06:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:06:28.549906 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2acc50-04b8-4d0c-b1ed-04b44c083660 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:28.587331 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1000ff53-a4b1-43ce-ac06-d3f6742025f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:28.587331 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000ff53-a4b1-43ce-ac06-d3f6742025f6 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:28.587331 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000ff53-a4b1-43ce-ac06-d3f6742025f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:28.587331 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000ff53-a4b1-43ce-ac06-d3f6742025f6 None admin] PATH_INFO: `/v3/services/network` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:28.587741 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1000ff53-a4b1-43ce-ac06-d3f6742025f6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:28.589241 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1000ff53-a4b1-43ce-ac06-d3f6742025f6 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:28.602715 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-1000ff53-a4b1-43ce-ac06-d3f6742025f6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 05:06:28.602715 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:28.602715 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 05:06:28.602715 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 05:06:28.602715 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 05:06:28.602715 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 05:06:28.602715 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 05:06:28.602715 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:06:28.602715 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 05:06:28.602715 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:28.602715 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 05:06:28.602715 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:28.602715 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:28.602715 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:28.602715 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:28.602715 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:28.602715 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:28.603801 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:28.603801 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:28.603801 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:28.603801 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:28.603801 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:28.603801 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:28.603801 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 05:06:28.603801 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 05:06:28.603801 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 05:06:28.603801 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 05:06:28.603801 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:28.603801 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:28.603801 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:28.603801 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:28.603801 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:28.603801 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:28.604747 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:28.604747 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:28.604747 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:28.604747 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:28.604747 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:28.604747 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:28.604747 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:28.604747 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:28.604747 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:28.604747 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:28.604747 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 05:06:28.604747 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:06:28.604747 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 05:06:28.604747 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:28.604747 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 342/708] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:06:28 2022] GET /identity/v3/services/network => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:28.610130 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1000ff53-a4b1-43ce-ac06-d3f6742025f6 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:28.673999 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2acc50-04b8-4d0c-b1ed-04b44c083660 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:28.713305 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd76ff8a-7089-44ba-bd36-a048a01deeb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:28.714292 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76ff8a-7089-44ba-bd36-a048a01deeb5 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:28.714749 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76ff8a-7089-44ba-bd36-a048a01deeb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:28.715217 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76ff8a-7089-44ba-bd36-a048a01deeb5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:28.728602 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1000ff53-a4b1-43ce-ac06-d3f6742025f6 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:28.752557 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:28.767972 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c51c23-354c-4c27-9cd2-ca8447d838f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:28.768579 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c51c23-354c-4c27-9cd2-ca8447d838f1 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:28.768852 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c51c23-354c-4c27-9cd2-ca8447d838f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:28.769116 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c51c23-354c-4c27-9cd2-ca8447d838f1 None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:28.769748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c51c23-354c-4c27-9cd2-ca8447d838f1 None admin] RBAC: Adding query filter params (name=network) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:28.770068 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c51c23-354c-4c27-9cd2-ca8447d838f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:28.771467 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c51c23-354c-4c27-9cd2-ca8447d838f1 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:28.776827 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 343/709] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Aug 31 05:06:28 2022] GET /identity/v3/services?name=network => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:06:28.777720 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c51c23-354c-4c27-9cd2-ca8447d838f1 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:28.794443 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:28.797484 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd76ff8a-7089-44ba-bd36-a048a01deeb5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:28.799444 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd76ff8a-7089-44ba-bd36-a048a01deeb5 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:28.892891 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c51c23-354c-4c27-9cd2-ca8447d838f1 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:28.934870 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 367/710] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:06:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 05:06:28.935760 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd76ff8a-7089-44ba-bd36-a048a01deeb5 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:28.937645 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f28bb-9424-4ddd-aadb-59fd816081a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:28.938326 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f28bb-9424-4ddd-aadb-59fd816081a0 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:28.938619 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f28bb-9424-4ddd-aadb-59fd816081a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:28.939047 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f28bb-9424-4ddd-aadb-59fd816081a0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:28.939539 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78f28bb-9424-4ddd-aadb-59fd816081a0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:28.941023 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78f28bb-9424-4ddd-aadb-59fd816081a0 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:28.950674 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-d78f28bb-9424-4ddd-aadb-59fd816081a0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 05:06:28.950674 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:28.950674 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 05:06:28.950674 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 05:06:28.950674 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 05:06:28.950674 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 05:06:28.950674 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 05:06:28.950674 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:06:28.950674 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 05:06:28.950674 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:28.950674 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 05:06:28.950674 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:28.950674 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:28.950674 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:28.950674 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:28.950674 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:28.950674 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:28.951789 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:28.951789 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:28.951789 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:28.951789 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:28.951789 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:28.951789 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:28.951789 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 05:06:28.951789 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 05:06:28.951789 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 05:06:28.951789 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 05:06:28.951789 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:28.951789 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:28.951789 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:28.951789 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:28.951789 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:28.951789 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:28.952679 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:28.952679 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:28.952679 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:28.952679 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:28.952679 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:28.952679 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:28.952679 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:28.952679 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:28.952679 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:28.952679 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:28.952679 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 05:06:28.952679 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:06:28.952679 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 05:06:28.952679 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:28.952679 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 344/711] 10.222.0.36 () {56 vars in 1129 bytes} [Wed Aug 31 05:06:28 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:28.953480 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f28bb-9424-4ddd-aadb-59fd816081a0 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:29.007917 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:29.046482 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:29.050903 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd76ff8a-7089-44ba-bd36-a048a01deeb5 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:29.063397 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f28bb-9424-4ddd-aadb-59fd816081a0 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:29.089291 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ba935-6e51-405a-9f8f-bb45962065fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:29.090180 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ba935-6e51-405a-9f8f-bb45962065fb None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:29.090603 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ba935-6e51-405a-9f8f-bb45962065fb None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:29.091051 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ba935-6e51-405a-9f8f-bb45962065fb None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:29.091964 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092ba935-6e51-405a-9f8f-bb45962065fb None admin] RBAC: Adding query filter params (type=network) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:29.092448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092ba935-6e51-405a-9f8f-bb45962065fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:29.094122 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092ba935-6e51-405a-9f8f-bb45962065fb None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:29.101330 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 368/712] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Aug 31 05:06:28 2022] GET /identity/v3/services?type=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:29.102633 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ba935-6e51-405a-9f8f-bb45962065fb None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:29.107701 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc3b80-db45-4a3e-8a07-b652cfabb433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:29.108436 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc3b80-db45-4a3e-8a07-b652cfabb433 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:29.108778 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc3b80-db45-4a3e-8a07-b652cfabb433 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:29.109113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc3b80-db45-4a3e-8a07-b652cfabb433 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:29.128537 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddc3b80-db45-4a3e-8a07-b652cfabb433 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:29.130434 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddc3b80-db45-4a3e-8a07-b652cfabb433 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:29.148658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-4ddc3b80-db45-4a3e-8a07-b652cfabb433 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:29.148658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:29.148658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:29.148658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:29.148658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:29.148658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:29.148658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:29.148658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:29.148658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:29.148658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:29.148658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:29.148658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:29.148658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:29.148658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:29.148658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:29.148658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:29.149696 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:29.149696 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:29.149696 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:29.149696 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:29.149696 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:29.149696 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:29.149696 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:29.149696 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:29.149696 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:29.149696 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:29.149696 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:29.149696 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:29.149696 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:29.149696 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:29.149696 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:29.149696 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:29.149696 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:29.150653 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:29.150653 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:29.150653 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:29.150653 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 345/713] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:29.155873 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc3b80-db45-4a3e-8a07-b652cfabb433 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:29.275102 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc3b80-db45-4a3e-8a07-b652cfabb433 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:29.276898 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:29.320457 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b234c-2192-43ff-83a3-f21a40a6e6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:29.320457 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b234c-2192-43ff-83a3-f21a40a6e6fe None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:29.320457 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b234c-2192-43ff-83a3-f21a40a6e6fe None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:29.320457 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b234c-2192-43ff-83a3-f21a40a6e6fe None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:29.320886 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460b234c-2192-43ff-83a3-f21a40a6e6fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:29.320886 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460b234c-2192-43ff-83a3-f21a40a6e6fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:29.322617 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460b234c-2192-43ff-83a3-f21a40a6e6fe None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:29.333943 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ba935-6e51-405a-9f8f-bb45962065fb None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:29.337751 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 346/714] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:29.344597 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b234c-2192-43ff-83a3-f21a40a6e6fe None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:29.401474 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a538c632-49f5-42f7-af3a-1ca66fa11878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:29.402489 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a538c632-49f5-42f7-af3a-1ca66fa11878 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:29.402996 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a538c632-49f5-42f7-af3a-1ca66fa11878 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:29.403465 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a538c632-49f5-42f7-af3a-1ca66fa11878 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:29.404490 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a538c632-49f5-42f7-af3a-1ca66fa11878 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:29.405011 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a538c632-49f5-42f7-af3a-1ca66fa11878 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:29.406751 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a538c632-49f5-42f7-af3a-1ca66fa11878 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:29.412093 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 369/715] 10.222.0.36 () {56 vars in 1124 bytes} [Wed Aug 31 05:06:29 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:29.418853 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a538c632-49f5-42f7-af3a-1ca66fa11878 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:29.451148 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:29.467979 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b234c-2192-43ff-83a3-f21a40a6e6fe None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:29.511131 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c61e1b9-8a81-4291-aaa7-cd20307e7471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:29.511748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61e1b9-8a81-4291-aaa7-cd20307e7471 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:29.512028 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61e1b9-8a81-4291-aaa7-cd20307e7471 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:29.512314 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61e1b9-8a81-4291-aaa7-cd20307e7471 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:29.521718 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c61e1b9-8a81-4291-aaa7-cd20307e7471 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:29.523223 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c61e1b9-8a81-4291-aaa7-cd20307e7471 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:29.536722 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-7c61e1b9-8a81-4291-aaa7-cd20307e7471 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:29.536722 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:29.536722 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:29.536722 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:29.536722 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:29.536722 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:29.536722 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:29.536722 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:29.536722 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:29.536722 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:29.536722 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:29.536722 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:29.536722 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 05:06:29.536722 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 05:06:29.536722 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 05:06:29.537625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 05:06:29.537625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:29.537625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:29.537625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:29.537625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:29.537625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:29.537625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:29.537625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:29.537625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:29.537625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:29.537625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:29.537625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:29.537625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:29.537625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:29.537625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:29.537625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:29.537625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:29.538882 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 05:06:29.538882 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 05:06:29.538882 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 05:06:29.538882 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 05:06:29.538882 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 05:06:29.538882 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 05:06:29.538882 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:29.538882 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:29.538882 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 347/716] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:06:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:29.543760 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a538c632-49f5-42f7-af3a-1ca66fa11878 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:29.554842 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c61e1b9-8a81-4291-aaa7-cd20307e7471 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:29.600808 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:29.615653 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623521eb-27c5-4d25-a388-ce1081e64df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:29.616273 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623521eb-27c5-4d25-a388-ce1081e64df8 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:29.616559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623521eb-27c5-4d25-a388-ce1081e64df8 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:29.616829 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623521eb-27c5-4d25-a388-ce1081e64df8 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:29.617466 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623521eb-27c5-4d25-a388-ce1081e64df8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:29.617764 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623521eb-27c5-4d25-a388-ce1081e64df8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:29.619490 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623521eb-27c5-4d25-a388-ce1081e64df8 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:29.630161 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 370/717] 10.222.0.36 () {56 vars in 1124 bytes} [Wed Aug 31 05:06:29 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:29.706385 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c61e1b9-8a81-4291-aaa7-cd20307e7471 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:29.743471 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca2d566-b7a1-4c05-b98e-a27765a54003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:29.744947 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2d566-b7a1-4c05-b98e-a27765a54003 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:29.745436 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2d566-b7a1-4c05-b98e-a27765a54003 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:29.745903 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2d566-b7a1-4c05-b98e-a27765a54003 None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:29.746965 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca2d566-b7a1-4c05-b98e-a27765a54003 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:29.747658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca2d566-b7a1-4c05-b98e-a27765a54003 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:29.749414 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca2d566-b7a1-4c05-b98e-a27765a54003 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:29.764944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 348/718] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 05:06:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:29.771004 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623521eb-27c5-4d25-a388-ce1081e64df8 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:29.777904 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b6311-fbeb-4345-82de-3b432245312f None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:29.778219 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b6311-fbeb-4345-82de-3b432245312f None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:29.778481 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b6311-fbeb-4345-82de-3b432245312f None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:29.779987 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 349/719] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 05:06:29.785539 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61830d5f-635a-4bda-89e1-5c9aef9fb50f None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:29.786175 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61830d5f-635a-4bda-89e1-5c9aef9fb50f None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:29.786704 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61830d5f-635a-4bda-89e1-5c9aef9fb50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:29.805963 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-61830d5f-635a-4bda-89e1-5c9aef9fb50f None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:29.807493 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-61830d5f-635a-4bda-89e1-5c9aef9fb50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:29.902880 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:29.914502 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623521eb-27c5-4d25-a388-ce1081e64df8 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:29.932942 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 350/720] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 05:06:29.952036 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e21391-ec0d-4a7c-89dd-cc075be8aeb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:29.952682 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e21391-ec0d-4a7c-89dd-cc075be8aeb1 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:29.952921 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e21391-ec0d-4a7c-89dd-cc075be8aeb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:29.953237 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e21391-ec0d-4a7c-89dd-cc075be8aeb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:29.953969 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e21391-ec0d-4a7c-89dd-cc075be8aeb1 None admin] RBAC: Adding query filter params (role.id=c3d5589dd2864be9a0f26f5802d874cd, scope.project.id=e01ce1b1282e44a39d659b87f120995e, user.id=f27665b1574a4db7b7a9771f7d775826) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:29.954306 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e21391-ec0d-4a7c-89dd-cc075be8aeb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:29.955800 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e21391-ec0d-4a7c-89dd-cc075be8aeb1 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:29.962328 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 371/721] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:06:29 2022] GET /identity/v3/role_assignments?user.id=f27665b1574a4db7b7a9771f7d775826&scope.project.id=e01ce1b1282e44a39d659b87f120995e&role.id=c3d5589dd2864be9a0f26f5802d874cd => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:30.223724 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61830d5f-635a-4bda-89e1-5c9aef9fb50f None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:30.243405 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:30.350027 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61830d5f-635a-4bda-89e1-5c9aef9fb50f None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:30.387130 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a4a68-391b-4f1c-b501-f29fc7046f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:30.387770 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a4a68-391b-4f1c-b501-f29fc7046f5f None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:30.388051 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a4a68-391b-4f1c-b501-f29fc7046f5f None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:30.388384 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a4a68-391b-4f1c-b501-f29fc7046f5f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:30.399139 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9a4a68-391b-4f1c-b501-f29fc7046f5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:06:30.403938 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-3b9a4a68-391b-4f1c-b501-f29fc7046f5f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 05:06:30.403938 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:30.403938 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:30.403938 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:30.403938 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:30.403938 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:30.403938 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:30.403938 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:30.403938 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:30.403938 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:30.403938 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:30.403938 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:30.403938 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:06:30.403938 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:06:30.403938 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:06:30.403938 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:06:30.404928 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:06:30.404928 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:06:30.404928 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:06:30.404928 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:06:30.404928 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:30.404928 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:30.404928 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:30.404928 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:30.404928 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:30.404928 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:30.404928 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:30.404928 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:30.404928 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:30.404928 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:30.404928 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:30.404928 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:30.406390 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:30.406390 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:30.406390 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:30.406390 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:30.406390 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:06:30.406390 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:06:30.406390 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:06:30.406390 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:06:30.406390 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:06:30.406390 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:06:30.406390 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 05:06:30.406390 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:30.406390 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 351/722] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 05:06:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:30.411217 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e21391-ec0d-4a7c-89dd-cc075be8aeb1 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:30.535794 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e21391-ec0d-4a7c-89dd-cc075be8aeb1 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:30.576361 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09e0c9-040b-4ce8-93c6-8b28c0bc5ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:30.576977 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09e0c9-040b-4ce8-93c6-8b28c0bc5ff4 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:30.577259 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09e0c9-040b-4ce8-93c6-8b28c0bc5ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:30.577534 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09e0c9-040b-4ce8-93c6-8b28c0bc5ff4 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:30.578238 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c09e0c9-040b-4ce8-93c6-8b28c0bc5ff4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:30.578542 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c09e0c9-040b-4ce8-93c6-8b28c0bc5ff4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:30.579940 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c09e0c9-040b-4ce8-93c6-8b28c0bc5ff4 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:30.589150 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 372/723] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Aug 31 05:06:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:30.594354 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a4a68-391b-4f1c-b501-f29fc7046f5f None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:30.691243 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:30.714994 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f969369-f7e9-4ae1-984f-bc5dbe5ae082 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:30.715405 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f969369-f7e9-4ae1-984f-bc5dbe5ae082 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:30.715683 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f969369-f7e9-4ae1-984f-bc5dbe5ae082 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:30.717827 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 373/724] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 05:06:30.723746 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5a646-ac77-4147-89c7-b2cbfc00d514 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:30.724034 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5a646-ac77-4147-89c7-b2cbfc00d514 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:30.724306 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5a646-ac77-4147-89c7-b2cbfc00d514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:30.742041 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a4a68-391b-4f1c-b501-f29fc7046f5f None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:30.745241 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d0e5a646-ac77-4147-89c7-b2cbfc00d514 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:30.746467 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d0e5a646-ac77-4147-89c7-b2cbfc00d514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:30.759090 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:30.823318 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c955f9-d957-47c4-ae47-c292fd0b5971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:30.824646 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c955f9-d957-47c4-ae47-c292fd0b5971 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:30.825214 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c955f9-d957-47c4-ae47-c292fd0b5971 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:30.825727 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c955f9-d957-47c4-ae47-c292fd0b5971 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:30.832217 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 374/725] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:06:30.854968 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c955f9-d957-47c4-ae47-c292fd0b5971 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:30.856713 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c955f9-d957-47c4-ae47-c292fd0b5971 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:30.882581 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-43c955f9-d957-47c4-ae47-c292fd0b5971 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:30.882581 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:30.882581 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:30.882581 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:30.882581 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:30.882581 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:30.882581 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:30.882581 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:30.882581 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:30.882581 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:30.882581 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:30.882581 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:30.882581 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:30.882581 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:30.882581 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:30.882581 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:30.883549 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:30.883549 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:30.883549 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:30.883549 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:30.883549 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:30.883549 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:30.883549 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:30.883549 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:30.883549 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:30.883549 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:30.883549 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:30.883549 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:30.883549 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:30.883549 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:30.883549 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:30.883549 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:30.883549 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:30.884447 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:30.884447 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:30.884447 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:30.886175 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 352/726] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:30.891976 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5a646-ac77-4147-89c7-b2cbfc00d514 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:31.016021 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5a646-ac77-4147-89c7-b2cbfc00d514 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:31.051532 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad4a2db-d6e7-4f4a-a0a3-e5b8e2477378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:31.052151 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad4a2db-d6e7-4f4a-a0a3-e5b8e2477378 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:31.052426 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad4a2db-d6e7-4f4a-a0a3-e5b8e2477378 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:31.052693 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad4a2db-d6e7-4f4a-a0a3-e5b8e2477378 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:31.053336 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad4a2db-d6e7-4f4a-a0a3-e5b8e2477378 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:31.053635 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad4a2db-d6e7-4f4a-a0a3-e5b8e2477378 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:31.055044 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad4a2db-d6e7-4f4a-a0a3-e5b8e2477378 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:31.066851 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 375/727] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:31.071996 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c955f9-d957-47c4-ae47-c292fd0b5971 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:31.194633 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad4a2db-d6e7-4f4a-a0a3-e5b8e2477378 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:31.206979 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:31.249922 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c955f9-d957-47c4-ae47-c292fd0b5971 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:31.295983 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:31.316138 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0956a816-177e-4006-b2cd-035e785e6761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:31.316766 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956a816-177e-4006-b2cd-035e785e6761 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:31.317038 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956a816-177e-4006-b2cd-035e785e6761 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:31.317307 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956a816-177e-4006-b2cd-035e785e6761 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:31.328203 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0956a816-177e-4006-b2cd-035e785e6761 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:31.329706 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0956a816-177e-4006-b2cd-035e785e6761 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:31.342838 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-0956a816-177e-4006-b2cd-035e785e6761 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 05:06:31.342838 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:31.342838 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:31.342838 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:31.342838 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:31.342838 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:31.342838 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:31.342838 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:31.342838 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:31.342838 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:31.342838 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:31.342838 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:31.342838 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 05:06:31.342838 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 05:06:31.342838 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 05:06:31.342838 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 05:06:31.343833 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:31.343833 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:31.343833 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 05:06:31.343833 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:31.343833 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 05:06:31.343833 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:31.343833 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:31.343833 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:31.343833 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:31.343833 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:31.343833 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:31.343833 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:31.343833 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:31.343833 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:31.343833 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:31.343833 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:31.343833 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:31.343833 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:31.345722 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:31.345722 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:31.345722 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 05:06:31.345722 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 05:06:31.345722 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 05:06:31.345722 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 05:06:31.345722 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 05:06:31.345722 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 05:06:31.345722 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 05:06:31.345722 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:31.345722 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 353/728] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Aug 31 05:06:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:31.349410 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0956a816-177e-4006-b2cd-035e785e6761 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:31.378767 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad4a2db-d6e7-4f4a-a0a3-e5b8e2477378 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:31.441505 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0603afff-4ff2-4bab-8b1b-ee2c7bb3c424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:31.442378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0603afff-4ff2-4bab-8b1b-ee2c7bb3c424 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:31.442650 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0603afff-4ff2-4bab-8b1b-ee2c7bb3c424 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:31.442912 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0603afff-4ff2-4bab-8b1b-ee2c7bb3c424 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:31.457216 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0603afff-4ff2-4bab-8b1b-ee2c7bb3c424 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:06:31.462795 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-0603afff-4ff2-4bab-8b1b-ee2c7bb3c424 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:06:31.462795 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:31.462795 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:31.462795 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:31.462795 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:31.462795 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:31.462795 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:31.462795 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:31.462795 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:31.462795 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:31.462795 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:31.462795 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:31.462795 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:06:31.462795 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:06:31.462795 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:06:31.462795 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:06:31.464761 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:06:31.464761 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:06:31.464761 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:06:31.464761 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:06:31.464761 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:31.464761 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:31.464761 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:31.464761 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:31.464761 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:31.464761 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:31.464761 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:31.464761 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:31.464761 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:31.464761 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:31.464761 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:31.464761 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:31.466407 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:31.466407 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:31.466407 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:31.466407 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:31.466407 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:06:31.466407 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:06:31.466407 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:06:31.466407 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:06:31.466407 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:06:31.466407 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:06:31.466407 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:06:31.466407 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:31.466407 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 376/729] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 05:06:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:06:31.477578 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0603afff-4ff2-4bab-8b1b-ee2c7bb3c424 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:31.501918 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0956a816-177e-4006-b2cd-035e785e6761 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:31.535967 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae9d257-7ea8-4404-b323-f7eb9f7f671c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:31.536572 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae9d257-7ea8-4404-b323-f7eb9f7f671c None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:31.536847 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae9d257-7ea8-4404-b323-f7eb9f7f671c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:31.537116 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae9d257-7ea8-4404-b323-f7eb9f7f671c None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:31.537847 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae9d257-7ea8-4404-b323-f7eb9f7f671c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:31.538179 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae9d257-7ea8-4404-b323-f7eb9f7f671c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:31.539626 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae9d257-7ea8-4404-b323-f7eb9f7f671c None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:31.561687 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 354/730] 10.222.0.36 () {56 vars in 1149 bytes} [Wed Aug 31 05:06:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:31.566185 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae9d257-7ea8-4404-b323-f7eb9f7f671c None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:31.606584 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:31.608935 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:31.670258 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae9d257-7ea8-4404-b323-f7eb9f7f671c None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:31.680573 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0603afff-4ff2-4bab-8b1b-ee2c7bb3c424 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:31.710863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1069aca4-88e7-4f9a-9c62-b45480f445a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:31.711931 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069aca4-88e7-4f9a-9c62-b45480f445a1 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:31.712581 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069aca4-88e7-4f9a-9c62-b45480f445a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:31.712996 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069aca4-88e7-4f9a-9c62-b45480f445a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:31.721263 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc68ed-a37b-437d-9322-303a9a677c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:31.721976 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc68ed-a37b-437d-9322-303a9a677c22 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:31.722348 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc68ed-a37b-437d-9322-303a9a677c22 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:31.722681 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc68ed-a37b-437d-9322-303a9a677c22 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:31.723417 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bc68ed-a37b-437d-9322-303a9a677c22 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:31.723779 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bc68ed-a37b-437d-9322-303a9a677c22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:31.725275 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bc68ed-a37b-437d-9322-303a9a677c22 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:31.738121 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 377/731] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:06:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:31.739524 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f00fac3-8c41-4384-9102-f7f3d89d8cc9 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:31.739890 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f00fac3-8c41-4384-9102-f7f3d89d8cc9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:31.740234 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f00fac3-8c41-4384-9102-f7f3d89d8cc9 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:31.741817 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 378/732] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:31.743973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f00fac3-8c41-4384-9102-f7f3d89d8cc9 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:31.803316 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1069aca4-88e7-4f9a-9c62-b45480f445a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:31.805605 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1069aca4-88e7-4f9a-9c62-b45480f445a1 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:31.856041 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f00fac3-8c41-4384-9102-f7f3d89d8cc9 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:31.864318 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:31.884975 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:31.893449 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef0ae7a-b1e7-4e8f-8b91-2163b600db8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:31.894262 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0ae7a-b1e7-4e8f-8b91-2163b600db8e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:31.894608 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0ae7a-b1e7-4e8f-8b91-2163b600db8e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:31.894931 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0ae7a-b1e7-4e8f-8b91-2163b600db8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:31.906390 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef0ae7a-b1e7-4e8f-8b91-2163b600db8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:31.907496 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef0ae7a-b1e7-4e8f-8b91-2163b600db8e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:31.922866 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-3ef0ae7a-b1e7-4e8f-8b91-2163b600db8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:31.922866 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:31.922866 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:31.922866 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:31.922866 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:31.922866 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:31.922866 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:31.922866 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:31.922866 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:31.922866 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:31.922866 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:31.922866 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:31.922866 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:31.922866 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:31.922866 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:31.922866 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:31.924131 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:31.924131 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:31.924131 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:31.924131 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:31.924131 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:31.924131 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:31.924131 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:31.924131 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:31.924131 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:31.924131 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:31.924131 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:31.924131 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:31.924131 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:31.924131 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:31.924131 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:31.924131 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:31.924131 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:31.925228 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:31.925228 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:31.925228 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:31.930444 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 379/733] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:31.931811 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e185b-d957-41c0-8446-47ca891d6913 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:31.932164 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e185b-d957-41c0-8446-47ca891d6913 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:31.932504 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e185b-d957-41c0-8446-47ca891d6913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:31.947235 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 355/734] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:06:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 05:06:31.948370 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1069aca4-88e7-4f9a-9c62-b45480f445a1 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:31.955552 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-de9e185b-d957-41c0-8446-47ca891d6913 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:31.956837 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-de9e185b-d957-41c0-8446-47ca891d6913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:32.027782 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 380/735] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:06:32.028838 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e185b-d957-41c0-8446-47ca891d6913 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:32.080418 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1069aca4-88e7-4f9a-9c62-b45480f445a1 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:32.137326 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9974b36e-31fe-4556-b1ff-480ca9ff2ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:32.137941 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974b36e-31fe-4556-b1ff-480ca9ff2ddc None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:32.138254 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e185b-d957-41c0-8446-47ca891d6913 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:32.138382 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974b36e-31fe-4556-b1ff-480ca9ff2ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:32.138562 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974b36e-31fe-4556-b1ff-480ca9ff2ddc None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:32.139212 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9974b36e-31fe-4556-b1ff-480ca9ff2ddc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:32.139533 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9974b36e-31fe-4556-b1ff-480ca9ff2ddc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:32.142029 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9974b36e-31fe-4556-b1ff-480ca9ff2ddc None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:32.150700 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:32.158299 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 356/736] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:32.159598 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b653008-fb5e-45d3-a08f-83835c72b7ae None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:32.159885 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b653008-fb5e-45d3-a08f-83835c72b7ae None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:32.160159 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b653008-fb5e-45d3-a08f-83835c72b7ae None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:32.161614 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 357/737] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:32.166861 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d19f1-0183-4b0b-b781-1343ab4ea1c5 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:32.167213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d19f1-0183-4b0b-b781-1343ab4ea1c5 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:32.167484 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d19f1-0183-4b0b-b781-1343ab4ea1c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:32.175597 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:32.186056 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36aa973-c666-48dd-a01f-3c4b4822b335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:32.186971 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36aa973-c666-48dd-a01f-3c4b4822b335 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:32.187473 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36aa973-c666-48dd-a01f-3c4b4822b335 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:32.187958 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36aa973-c666-48dd-a01f-3c4b4822b335 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:32.192406 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-2f4d19f1-0183-4b0b-b781-1343ab4ea1c5 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:32.193652 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-2f4d19f1-0183-4b0b-b781-1343ab4ea1c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:32.199590 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36aa973-c666-48dd-a01f-3c4b4822b335 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:32.201440 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36aa973-c666-48dd-a01f-3c4b4822b335 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:32.216921 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-a36aa973-c666-48dd-a01f-3c4b4822b335 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:32.216921 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:32.216921 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:32.216921 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:32.216921 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:32.216921 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:32.216921 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:32.216921 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:32.216921 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:32.216921 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:32.216921 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:32.216921 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:32.216921 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:32.216921 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:32.216921 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:32.216921 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:32.217909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:32.217909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:32.217909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:32.217909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:32.217909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:32.217909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:32.217909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:32.217909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:32.217909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:32.217909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:32.217909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:32.217909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:32.217909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:32.217909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:32.217909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:32.217909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:32.217909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:32.218844 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:32.218844 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:32.218844 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:32.220943 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 381/738] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:32.221932 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36aa973-c666-48dd-a01f-3c4b4822b335 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:32.278203 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 358/739] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:06:32.279561 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d19f1-0183-4b0b-b781-1343ab4ea1c5 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:32.351645 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36aa973-c666-48dd-a01f-3c4b4822b335 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:32.387957 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dc39e7-2409-400d-87a4-16624545895b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:32.388623 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc39e7-2409-400d-87a4-16624545895b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:32.388885 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc39e7-2409-400d-87a4-16624545895b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:32.389156 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc39e7-2409-400d-87a4-16624545895b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:32.397453 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dc39e7-2409-400d-87a4-16624545895b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:32.399376 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dc39e7-2409-400d-87a4-16624545895b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:32.411048 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-95dc39e7-2409-400d-87a4-16624545895b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 05:06:32.411048 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:32.411048 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:32.411048 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:32.411048 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:32.411048 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:32.411048 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:32.411048 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:32.411048 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:32.411048 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:32.411048 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:32.411048 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:32.411048 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 05:06:32.411048 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 05:06:32.411048 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 05:06:32.411048 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 05:06:32.411973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:32.411973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:32.411973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 05:06:32.411973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:32.411973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 05:06:32.411973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:32.411973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:32.411973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:32.411973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:32.411973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:32.411973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:32.411973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:32.411973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:32.411973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:32.411973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:32.411973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:32.411973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:32.411973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:32.412917 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:32.412917 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:32.412917 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 05:06:32.412917 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 05:06:32.412917 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 05:06:32.412917 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 05:06:32.412917 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 05:06:32.412917 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 05:06:32.412917 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 05:06:32.412917 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:32.412917 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 382/740] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 05:06:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:32.412917 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dc39e7-2409-400d-87a4-16624545895b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:32.418868 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d19f1-0183-4b0b-b781-1343ab4ea1c5 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:32.440121 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:32.450287 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:32.456185 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049db8c4-1bbc-4f6d-8a91-b775f525beeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:32.457730 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049db8c4-1bbc-4f6d-8a91-b775f525beeb None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:32.458259 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049db8c4-1bbc-4f6d-8a91-b775f525beeb None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:32.458739 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049db8c4-1bbc-4f6d-8a91-b775f525beeb None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:32.460856 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049db8c4-1bbc-4f6d-8a91-b775f525beeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:32.461481 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049db8c4-1bbc-4f6d-8a91-b775f525beeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:32.463664 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049db8c4-1bbc-4f6d-8a91-b775f525beeb None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:32.482049 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 359/741] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:32.482049 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b21b6-5e1c-4010-8296-d2bd3c433593 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:32.482049 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b21b6-5e1c-4010-8296-d2bd3c433593 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:32.482049 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b21b6-5e1c-4010-8296-d2bd3c433593 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:32.482049 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 360/742] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:32.482049 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b21b6-5e1c-4010-8296-d2bd3c433593 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:32.528762 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dc39e7-2409-400d-87a4-16624545895b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:32.562322 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9baaa-7b9b-4476-9936-b9da3d98b6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:32.562923 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9baaa-7b9b-4476-9936-b9da3d98b6e9 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:32.563196 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9baaa-7b9b-4476-9936-b9da3d98b6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:32.563469 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9baaa-7b9b-4476-9936-b9da3d98b6e9 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:32.564114 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da9baaa-7b9b-4476-9936-b9da3d98b6e9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:32.565487 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da9baaa-7b9b-4476-9936-b9da3d98b6e9 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:32.582644 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 383/743] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Aug 31 05:06:32 2022] POST /identity/v3/services => generated 240 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:06:32.583866 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bfbdd9-95d3-4347-85e1-5769b8137a0f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:32.584165 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bfbdd9-95d3-4347-85e1-5769b8137a0f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:32.584420 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bfbdd9-95d3-4347-85e1-5769b8137a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:32.603867 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-36bfbdd9-95d3-4347-85e1-5769b8137a0f None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:32.605110 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-36bfbdd9-95d3-4347-85e1-5769b8137a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:32.605241 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b21b6-5e1c-4010-8296-d2bd3c433593 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:32.653401 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebf8912-3312-4666-a945-b91c0feb5b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:32.653401 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf8912-3312-4666-a945-b91c0feb5b98 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:32.653401 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf8912-3312-4666-a945-b91c0feb5b98 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:32.653401 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf8912-3312-4666-a945-b91c0feb5b98 None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:32.654062 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebf8912-3312-4666-a945-b91c0feb5b98 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:32.654062 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebf8912-3312-4666-a945-b91c0feb5b98 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:32.654575 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebf8912-3312-4666-a945-b91c0feb5b98 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:32.678119 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 361/744] 10.222.0.36 () {56 vars in 1135 bytes} [Wed Aug 31 05:06:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:32.683736 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebf8912-3312-4666-a945-b91c0feb5b98 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:32.690265 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 384/745] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:32 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 05:06:32.690265 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bfbdd9-95d3-4347-85e1-5769b8137a0f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:32.707307 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:32.746835 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:32.807828 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebf8912-3312-4666-a945-b91c0feb5b98 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:32.814759 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bfbdd9-95d3-4347-85e1-5769b8137a0f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:32.847261 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b6ea72-53b6-48a2-a939-7c308244b88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:32.848249 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b6ea72-53b6-48a2-a939-7c308244b88e None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:32.848748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b6ea72-53b6-48a2-a939-7c308244b88e None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:32.849210 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b6ea72-53b6-48a2-a939-7c308244b88e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:32.850661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9e31d-7f79-42ae-944d-727e2f2ff7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:32.851274 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9e31d-7f79-42ae-944d-727e2f2ff7c4 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:32.851558 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9e31d-7f79-42ae-944d-727e2f2ff7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:32.851821 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9e31d-7f79-42ae-944d-727e2f2ff7c4 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:32.852572 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe9e31d-7f79-42ae-944d-727e2f2ff7c4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:32.854043 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe9e31d-7f79-42ae-944d-727e2f2ff7c4 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:32.857794 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b6ea72-53b6-48a2-a939-7c308244b88e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:32.859651 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b6ea72-53b6-48a2-a939-7c308244b88e None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:32.871440 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-22b6ea72-53b6-48a2-a939-7c308244b88e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:32.871440 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:32.871440 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:32.871440 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:32.871440 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:32.871440 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:32.871440 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:32.871440 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:32.871440 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:32.871440 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:32.871440 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:32.871440 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:32.871440 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 05:06:32.871440 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 05:06:32.871440 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 05:06:32.872976 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 05:06:32.872976 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:32.872976 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:32.872976 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:32.872976 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:32.872976 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:32.872976 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:32.872976 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:32.872976 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:32.872976 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:32.872976 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:32.872976 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:32.872976 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:32.872976 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:32.872976 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:32.872976 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:32.872976 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:32.873882 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 05:06:32.873882 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 05:06:32.873882 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 05:06:32.873882 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 05:06:32.873882 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 05:06:32.873882 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 05:06:32.873882 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:32.873882 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:32.875313 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 385/747] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Aug 31 05:06:32 2022] POST /identity/v3/services => generated 251 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:06:32.876034 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 362/747] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:06:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:32.876661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9e31d-7f79-42ae-944d-727e2f2ff7c4 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:32.878721 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b6ea72-53b6-48a2-a939-7c308244b88e None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:33.001701 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:33.043923 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b6ea72-53b6-48a2-a939-7c308244b88e None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:33.051879 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:33.056311 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9e31d-7f79-42ae-944d-727e2f2ff7c4 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:33.090759 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aef56d-6b8d-4d7c-80aa-d3932847e955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:33.091675 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aef56d-6b8d-4d7c-80aa-d3932847e955 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:33.092206 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aef56d-6b8d-4d7c-80aa-d3932847e955 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:33.092477 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aef56d-6b8d-4d7c-80aa-d3932847e955 None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:33.093181 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aef56d-6b8d-4d7c-80aa-d3932847e955 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:33.093483 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aef56d-6b8d-4d7c-80aa-d3932847e955 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:33.094968 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aef56d-6b8d-4d7c-80aa-d3932847e955 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:33.095863 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada05e91-9fa6-4001-bb5c-51c8bd34fefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:33.096529 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada05e91-9fa6-4001-bb5c-51c8bd34fefb None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:33.096805 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada05e91-9fa6-4001-bb5c-51c8bd34fefb None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:33.097070 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada05e91-9fa6-4001-bb5c-51c8bd34fefb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:33.110411 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 363/748] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 05:06:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:06:33.111423 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aef56d-6b8d-4d7c-80aa-d3932847e955 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:33.169557 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada05e91-9fa6-4001-bb5c-51c8bd34fefb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:33.171543 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada05e91-9fa6-4001-bb5c-51c8bd34fefb None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:33.214254 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aef56d-6b8d-4d7c-80aa-d3932847e955 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:33.249284 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5201c54-7898-4b34-9314-f0e56cef7367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:33.249914 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5201c54-7898-4b34-9314-f0e56cef7367 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:33.250216 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5201c54-7898-4b34-9314-f0e56cef7367 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:33.250485 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5201c54-7898-4b34-9314-f0e56cef7367 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:33.261129 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5201c54-7898-4b34-9314-f0e56cef7367 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:06:33.265920 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-b5201c54-7898-4b34-9314-f0e56cef7367 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:06:33.265920 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:33.265920 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:33.265920 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:33.265920 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:33.265920 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:33.265920 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:33.265920 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:33.265920 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:33.265920 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:33.265920 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:33.265920 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:33.265920 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:06:33.265920 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:06:33.265920 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:06:33.265920 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:06:33.266908 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:06:33.266908 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:06:33.266908 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:06:33.266908 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:06:33.266908 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:33.266908 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:33.266908 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:33.266908 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:33.266908 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:33.266908 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:33.266908 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:33.266908 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:33.266908 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:33.266908 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:33.266908 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:33.266908 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:33.267790 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:33.267790 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:33.267790 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:33.267790 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:33.267790 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:06:33.267790 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:06:33.267790 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:06:33.267790 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:06:33.267790 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:06:33.267790 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:06:33.267790 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:06:33.267790 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:33.268442 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:33.271932 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 364/749] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 05:06:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:33.272804 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5201c54-7898-4b34-9314-f0e56cef7367 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:33.285154 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 386/750] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:06:32 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 05:06:33.286100 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada05e91-9fa6-4001-bb5c-51c8bd34fefb None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:33.306179 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:33.381072 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5201c54-7898-4b34-9314-f0e56cef7367 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:33.395970 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada05e91-9fa6-4001-bb5c-51c8bd34fefb None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:33.416247 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0376a07d-ffec-411f-b495-bdd206c6d392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:33.416847 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376a07d-ffec-411f-b495-bdd206c6d392 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:33.417120 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376a07d-ffec-411f-b495-bdd206c6d392 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:33.417386 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376a07d-ffec-411f-b495-bdd206c6d392 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:33.418139 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0376a07d-ffec-411f-b495-bdd206c6d392 None admin] RBAC: Adding query filter params (role.id=207bf98e31774cc0b21249de6d96155b, scope.project.id=e01ce1b1282e44a39d659b87f120995e, user.id=09bf6db88ce8494a937211a814c9ff84) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:33.418442 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0376a07d-ffec-411f-b495-bdd206c6d392 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:33.419934 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0376a07d-ffec-411f-b495-bdd206c6d392 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:33.426085 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 365/751] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:06:33 2022] GET /identity/v3/role_assignments?user.id=09bf6db88ce8494a937211a814c9ff84&scope.project.id=e01ce1b1282e44a39d659b87f120995e&role.id=207bf98e31774cc0b21249de6d96155b => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:33.426756 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0376a07d-ffec-411f-b495-bdd206c6d392 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:33.434654 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4dc50e-afc8-496a-8082-673e9f3cb804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:33.435777 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4dc50e-afc8-496a-8082-673e9f3cb804 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:33.436250 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4dc50e-afc8-496a-8082-673e9f3cb804 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:33.436734 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4dc50e-afc8-496a-8082-673e9f3cb804 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:33.437618 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4dc50e-afc8-496a-8082-673e9f3cb804 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:33.438170 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4dc50e-afc8-496a-8082-673e9f3cb804 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:33.439781 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4dc50e-afc8-496a-8082-673e9f3cb804 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:33.450357 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 387/752] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:06:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:33.456338 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4dc50e-afc8-496a-8082-673e9f3cb804 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:33.526910 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:33.607701 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:33.617244 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0376a07d-ffec-411f-b495-bdd206c6d392 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:33.628858 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4dc50e-afc8-496a-8082-673e9f3cb804 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:33.667939 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf1bf5-2947-4deb-9076-7ff314975f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:33.668829 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf1bf5-2947-4deb-9076-7ff314975f68 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:33.669225 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf1bf5-2947-4deb-9076-7ff314975f68 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:33.669674 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf1bf5-2947-4deb-9076-7ff314975f68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:33.674147 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cfb81c-0cae-4c86-a907-14dab8f4a38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:33.674958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfb81c-0cae-4c86-a907-14dab8f4a38d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:33.675368 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfb81c-0cae-4c86-a907-14dab8f4a38d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:33.675810 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfb81c-0cae-4c86-a907-14dab8f4a38d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:33.699598 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cfb81c-0cae-4c86-a907-14dab8f4a38d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:33.700602 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf1bf5-2947-4deb-9076-7ff314975f68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:33.702276 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cfb81c-0cae-4c86-a907-14dab8f4a38d None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:33.703597 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf1bf5-2947-4deb-9076-7ff314975f68 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:33.725554 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-d5cfb81c-0cae-4c86-a907-14dab8f4a38d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:33.725554 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:33.725554 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:33.725554 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:33.725554 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:33.725554 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:33.725554 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:33.725554 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:33.725554 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:33.725554 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:33.725554 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:33.725554 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:33.725554 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:33.725554 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:33.725554 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:33.725554 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:33.726625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:33.726625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:33.726625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:33.726625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:33.726625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:33.726625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:33.726625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:33.726625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:33.726625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:33.726625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:33.726625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:33.726625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:33.726625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:33.726625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:33.726625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:33.726625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:33.726625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:33.727523 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:33.727523 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:33.727523 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:33.734938 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-9aaf1bf5-2947-4deb-9076-7ff314975f68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:33.734938 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:33.734938 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:33.734938 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:33.734938 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:33.734938 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:33.734938 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:33.734938 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:33.734938 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:33.734938 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:33.734938 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:33.734938 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:33.734938 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:33.734938 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:33.734938 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:33.734938 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:33.735869 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:33.735869 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:33.735869 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:33.735869 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:33.735869 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:33.735869 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:33.735869 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:33.735869 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:33.735869 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:33.735869 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:33.735869 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:33.735869 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:33.735869 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:33.735869 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:33.735869 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:33.735869 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:33.735869 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:33.736760 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:33.736760 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:33.736760 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:33.738967 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 366/753] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:33.740685 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 388/754] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:33.745280 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cfb81c-0cae-4c86-a907-14dab8f4a38d None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:33.746222 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf1bf5-2947-4deb-9076-7ff314975f68 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:33.877742 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:33.908151 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf1bf5-2947-4deb-9076-7ff314975f68 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:33.944763 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cfb81c-0cae-4c86-a907-14dab8f4a38d None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:33.960123 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de36a4c-cb9e-43f4-b15e-a3e2389be5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:33.960764 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de36a4c-cb9e-43f4-b15e-a3e2389be5c1 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:33.961064 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de36a4c-cb9e-43f4-b15e-a3e2389be5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:33.961362 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de36a4c-cb9e-43f4-b15e-a3e2389be5c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:33.962040 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de36a4c-cb9e-43f4-b15e-a3e2389be5c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:33.962334 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de36a4c-cb9e-43f4-b15e-a3e2389be5c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:33.963722 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de36a4c-cb9e-43f4-b15e-a3e2389be5c1 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:33.980198 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 389/755] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:33.985501 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de36a4c-cb9e-43f4-b15e-a3e2389be5c1 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:34.018707 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d32bad-c663-4fa7-91fb-1d3f78e4a492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:34.019302 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d32bad-c663-4fa7-91fb-1d3f78e4a492 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:34.019574 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d32bad-c663-4fa7-91fb-1d3f78e4a492 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:34.019833 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d32bad-c663-4fa7-91fb-1d3f78e4a492 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:34.020476 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d32bad-c663-4fa7-91fb-1d3f78e4a492 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:34.020771 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d32bad-c663-4fa7-91fb-1d3f78e4a492 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:34.022171 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d32bad-c663-4fa7-91fb-1d3f78e4a492 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:34.042706 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 367/756] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:34.047032 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d32bad-c663-4fa7-91fb-1d3f78e4a492 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:34.051307 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:34.157808 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de36a4c-cb9e-43f4-b15e-a3e2389be5c1 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:34.224628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0252de-7c40-4b07-a7f5-dc85eb01ba83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:34.225168 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0252de-7c40-4b07-a7f5-dc85eb01ba83 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:34.225442 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0252de-7c40-4b07-a7f5-dc85eb01ba83 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:34.225705 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0252de-7c40-4b07-a7f5-dc85eb01ba83 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:34.236846 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0252de-7c40-4b07-a7f5-dc85eb01ba83 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:34.240096 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d32bad-c663-4fa7-91fb-1d3f78e4a492 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:34.243924 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0252de-7c40-4b07-a7f5-dc85eb01ba83 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:34.271081 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-ce0252de-7c40-4b07-a7f5-dc85eb01ba83 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 05:06:34.271081 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:34.271081 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:34.271081 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:34.271081 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:34.271081 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:34.271081 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:34.271081 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:34.271081 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:34.271081 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:34.271081 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:34.271081 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:34.271081 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 05:06:34.271081 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 05:06:34.271081 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 05:06:34.271081 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 05:06:34.272091 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:34.272091 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:34.272091 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 05:06:34.272091 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:34.272091 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 05:06:34.272091 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:34.272091 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:34.272091 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:34.272091 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:34.272091 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:34.272091 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:34.272091 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:34.272091 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:34.272091 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:34.272091 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:34.272091 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:34.272091 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:34.272091 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:34.273065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:34.273065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:34.273065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 05:06:34.273065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 05:06:34.273065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 05:06:34.273065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 05:06:34.273065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 05:06:34.273065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 05:06:34.273065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 05:06:34.273065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:34.273065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 390/757] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Aug 31 05:06:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:34.278939 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0252de-7c40-4b07-a7f5-dc85eb01ba83 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:34.284319 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e736f1b1-6c14-4893-ba78-6516dbab7089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:34.285181 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e736f1b1-6c14-4893-ba78-6516dbab7089 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:34.285646 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e736f1b1-6c14-4893-ba78-6516dbab7089 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:34.286160 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e736f1b1-6c14-4893-ba78-6516dbab7089 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:34.288435 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:34.295036 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e736f1b1-6c14-4893-ba78-6516dbab7089 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:34.296740 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e736f1b1-6c14-4893-ba78-6516dbab7089 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:34.309267 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-e736f1b1-6c14-4893-ba78-6516dbab7089 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:34.309267 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:34.309267 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:34.309267 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:34.309267 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:34.309267 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:34.309267 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:34.309267 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:34.309267 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:34.309267 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:34.309267 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:34.309267 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:34.309267 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 05:06:34.309267 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 05:06:34.309267 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 05:06:34.310270 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 05:06:34.310270 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:34.310270 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:34.310270 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:34.310270 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:34.310270 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:34.310270 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:34.310270 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:34.310270 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:34.310270 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:34.310270 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:34.310270 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:34.310270 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:34.310270 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:34.310270 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:34.310270 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:34.310270 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:34.311166 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 05:06:34.311166 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 05:06:34.311166 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 05:06:34.311166 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 05:06:34.311166 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 05:06:34.311166 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 05:06:34.311166 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:34.311166 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:34.313627 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 368/758] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:06:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:34.318861 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e736f1b1-6c14-4893-ba78-6516dbab7089 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:34.419188 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0252de-7c40-4b07-a7f5-dc85eb01ba83 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:34.453077 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97877133-ba5e-430c-b8b9-57f52dc3e94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:34.453711 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97877133-ba5e-430c-b8b9-57f52dc3e94f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:34.453984 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97877133-ba5e-430c-b8b9-57f52dc3e94f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:34.454274 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97877133-ba5e-430c-b8b9-57f52dc3e94f None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:34.456789 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97877133-ba5e-430c-b8b9-57f52dc3e94f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:34.456789 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97877133-ba5e-430c-b8b9-57f52dc3e94f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:34.456789 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97877133-ba5e-430c-b8b9-57f52dc3e94f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:34.474309 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 391/759] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Aug 31 05:06:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:34.475971 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:34.483050 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97877133-ba5e-430c-b8b9-57f52dc3e94f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:34.532108 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e736f1b1-6c14-4893-ba78-6516dbab7089 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:34.565971 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466d2dee-bdf7-4f5d-80cc-56423dea6165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:34.566979 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d2dee-bdf7-4f5d-80cc-56423dea6165 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:34.567469 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d2dee-bdf7-4f5d-80cc-56423dea6165 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:34.567926 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d2dee-bdf7-4f5d-80cc-56423dea6165 None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:34.568809 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466d2dee-bdf7-4f5d-80cc-56423dea6165 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:34.569324 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466d2dee-bdf7-4f5d-80cc-56423dea6165 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:34.570995 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466d2dee-bdf7-4f5d-80cc-56423dea6165 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:34.584957 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 369/760] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 05:06:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:06:34.591055 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466d2dee-bdf7-4f5d-80cc-56423dea6165 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:34.600261 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:34.639453 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97877133-ba5e-430c-b8b9-57f52dc3e94f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:34.675217 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7a6fc6-9a91-4fb0-85c7-2970e19c5b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:34.675850 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a6fc6-9a91-4fb0-85c7-2970e19c5b8b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:34.676127 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a6fc6-9a91-4fb0-85c7-2970e19c5b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:34.676410 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a6fc6-9a91-4fb0-85c7-2970e19c5b8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:34.687423 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7a6fc6-9a91-4fb0-85c7-2970e19c5b8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:34.688942 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7a6fc6-9a91-4fb0-85c7-2970e19c5b8b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:34.704137 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-6d7a6fc6-9a91-4fb0-85c7-2970e19c5b8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:34.704137 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:34.704137 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:34.704137 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:34.704137 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:34.704137 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:34.704137 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:34.704137 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:34.704137 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:34.704137 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:34.704137 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:34.704137 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:34.704137 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:34.704137 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:34.704137 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:34.704137 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:34.705148 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:34.705148 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:34.705148 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:34.705148 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:34.705148 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:34.705148 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:34.705148 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:34.705148 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:34.705148 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:34.705148 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:34.705148 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:34.705148 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:34.705148 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:34.705148 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:34.705148 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:34.705148 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:34.705148 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:34.706105 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:34.706105 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:34.706105 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:34.706105 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 392/761] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:34.713548 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7a6fc6-9a91-4fb0-85c7-2970e19c5b8b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:34.820661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7a6fc6-9a91-4fb0-85c7-2970e19c5b8b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:34.832310 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466d2dee-bdf7-4f5d-80cc-56423dea6165 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:34.834960 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:34.860085 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7ad544-2372-47a8-8869-9e3bcd28280c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:34.860939 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ad544-2372-47a8-8869-9e3bcd28280c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:34.861354 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ad544-2372-47a8-8869-9e3bcd28280c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:34.861590 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:34.861765 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ad544-2372-47a8-8869-9e3bcd28280c None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:34.862763 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7ad544-2372-47a8-8869-9e3bcd28280c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:34.863066 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7ad544-2372-47a8-8869-9e3bcd28280c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:34.864589 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7ad544-2372-47a8-8869-9e3bcd28280c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:34.877257 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41843791-33e2-4e93-b754-0b53b66b2ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:34.878201 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 393/762] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:34.878427 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41843791-33e2-4e93-b754-0b53b66b2ab1 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:34.878868 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41843791-33e2-4e93-b754-0b53b66b2ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:34.879403 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb507ea-2ea6-492b-b3fd-987e0a3d7374 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:34.879598 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41843791-33e2-4e93-b754-0b53b66b2ab1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:34.879695 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb507ea-2ea6-492b-b3fd-987e0a3d7374 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:34.879933 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb507ea-2ea6-492b-b3fd-987e0a3d7374 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:34.880764 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41843791-33e2-4e93-b754-0b53b66b2ab1 None admin] RBAC: Adding query filter params (role.id=c3d5589dd2864be9a0f26f5802d874cd, scope.project.id=e01ce1b1282e44a39d659b87f120995e, user.id=d9b82080a73d4c4a8ec08b0e67d6d11c) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:34.881239 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41843791-33e2-4e93-b754-0b53b66b2ab1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:34.883111 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41843791-33e2-4e93-b754-0b53b66b2ab1 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:34.885239 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 394/763] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:34.886069 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb507ea-2ea6-492b-b3fd-987e0a3d7374 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:34.891028 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 370/764] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:06:34 2022] GET /identity/v3/role_assignments?user.id=d9b82080a73d4c4a8ec08b0e67d6d11c&scope.project.id=e01ce1b1282e44a39d659b87f120995e&role.id=c3d5589dd2864be9a0f26f5802d874cd => generated 259 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:34.898051 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac5dbb-5db6-4300-b1a1-46d3e29bc40c None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:34.898880 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac5dbb-5db6-4300-b1a1-46d3e29bc40c None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:34.898880 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac5dbb-5db6-4300-b1a1-46d3e29bc40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:34.939100 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-b3ac5dbb-5db6-4300-b1a1-46d3e29bc40c None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:34.940310 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b3ac5dbb-5db6-4300-b1a1-46d3e29bc40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:35.035606 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 371/765] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:06:35.042587 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f6376-4e41-4eb0-ad32-c6163913aaa4 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:35.042897 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f6376-4e41-4eb0-ad32-c6163913aaa4 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:35.043141 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f6376-4e41-4eb0-ad32-c6163913aaa4 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:35.044977 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 372/766] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:35 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:35.050235 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb3345-fed3-4ca0-a02e-fd2cbb3150dd None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:35.050528 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb3345-fed3-4ca0-a02e-fd2cbb3150dd None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:35.050800 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb3345-fed3-4ca0-a02e-fd2cbb3150dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:35.056448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb507ea-2ea6-492b-b3fd-987e0a3d7374 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:35.076482 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-5ebb3345-fed3-4ca0-a02e-fd2cbb3150dd None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:35.078968 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-5ebb3345-fed3-4ca0-a02e-fd2cbb3150dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:35.099342 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d8afb2-3271-403f-bf55-d6c09921042f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:35.099983 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d8afb2-3271-403f-bf55-d6c09921042f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:35.100289 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d8afb2-3271-403f-bf55-d6c09921042f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:35.100566 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d8afb2-3271-403f-bf55-d6c09921042f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:35.115964 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d8afb2-3271-403f-bf55-d6c09921042f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:35.115964 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d8afb2-3271-403f-bf55-d6c09921042f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:35.137141 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-c3d8afb2-3271-403f-bf55-d6c09921042f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:35.137141 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:35.137141 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:35.137141 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:35.137141 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:35.137141 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:35.137141 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:35.137141 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:35.137141 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:35.137141 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:35.137141 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:35.137141 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:35.137141 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 05:06:35.137141 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 05:06:35.137141 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 05:06:35.138138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 05:06:35.138138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:35.138138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:35.138138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:35.138138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:35.138138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:35.138138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:35.138138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:35.138138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:35.138138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:35.138138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:35.138138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:35.138138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:35.138138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:35.138138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:35.138138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:35.138138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:35.139065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 05:06:35.139065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 05:06:35.139065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 05:06:35.139065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 05:06:35.139065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 05:06:35.139065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 05:06:35.139065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:35.139065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:35.141239 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 395/767] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:06:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:35.146641 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d8afb2-3271-403f-bf55-d6c09921042f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:35.184426 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:35.215707 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 373/768] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:06:35.259208 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d8afb2-3271-403f-bf55-d6c09921042f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:35.300197 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262124ef-56f1-4c92-aea1-5ce48e977950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:35.300801 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262124ef-56f1-4c92-aea1-5ce48e977950 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:35.301072 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262124ef-56f1-4c92-aea1-5ce48e977950 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:35.301381 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262124ef-56f1-4c92-aea1-5ce48e977950 None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:35.302099 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262124ef-56f1-4c92-aea1-5ce48e977950 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:35.302402 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262124ef-56f1-4c92-aea1-5ce48e977950 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:35.303869 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262124ef-56f1-4c92-aea1-5ce48e977950 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:35.316580 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 396/769] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 05:06:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:06:35.322347 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb3345-fed3-4ca0-a02e-fd2cbb3150dd None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:35.351205 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262124ef-56f1-4c92-aea1-5ce48e977950 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:35.415490 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:35.445755 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:35.459630 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262124ef-56f1-4c92-aea1-5ce48e977950 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:35.511162 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb3345-fed3-4ca0-a02e-fd2cbb3150dd None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:35.525743 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc748e4-31bf-4aaf-8408-87a44cdab5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:35.526411 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc748e4-31bf-4aaf-8408-87a44cdab5f5 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:35.526686 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc748e4-31bf-4aaf-8408-87a44cdab5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:35.526958 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc748e4-31bf-4aaf-8408-87a44cdab5f5 None admin] PATH_INFO: `/v3/services/network` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:35.527534 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc748e4-31bf-4aaf-8408-87a44cdab5f5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:35.528929 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc748e4-31bf-4aaf-8408-87a44cdab5f5 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:35.540005 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-6dc748e4-31bf-4aaf-8408-87a44cdab5f5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 05:06:35.540005 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:35.540005 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 05:06:35.540005 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 05:06:35.540005 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 05:06:35.540005 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 05:06:35.540005 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 05:06:35.540005 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:06:35.540005 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 05:06:35.540005 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:35.540005 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 05:06:35.540005 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:35.540005 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:35.540005 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:35.540005 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:35.540005 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:35.540005 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:35.541057 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:35.541057 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:35.541057 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:35.541057 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:35.541057 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:35.541057 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:35.541057 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 05:06:35.541057 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 05:06:35.541057 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 05:06:35.541057 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 05:06:35.541057 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:35.541057 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:35.541057 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:35.541057 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:35.541057 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:35.541057 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:35.542447 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:35.542447 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:35.542447 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:35.542447 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:35.542447 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:35.542447 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:35.542447 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:35.542447 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:35.542447 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:35.542447 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:35.542447 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 05:06:35.542447 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:06:35.542447 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 05:06:35.542447 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:35.542447 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 397/770] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:06:35 2022] GET /identity/v3/services/network => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:06:35.543272 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc748e4-31bf-4aaf-8408-87a44cdab5f5 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:35.556464 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a72079-243a-4d98-931d-f2e545195681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:35.557570 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a72079-243a-4d98-931d-f2e545195681 None admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:35.557858 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a72079-243a-4d98-931d-f2e545195681 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:35.558160 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a72079-243a-4d98-931d-f2e545195681 None admin] PATH_INFO: `/v3/projects/e01ce1b1282e44a39d659b87f120995e/users/f27665b1574a4db7b7a9771f7d775826/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:35.560744 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a72079-243a-4d98-931d-f2e545195681 None admin] RBAC: Authorizing `identity:create_grant(project_id=e01ce1b1282e44a39d659b87f120995e, user_id=f27665b1574a4db7b7a9771f7d775826, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:35.562477 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a72079-243a-4d98-931d-f2e545195681 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:35.584352 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 374/771] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 05:06:35 2022] PUT /identity/v3/projects/e01ce1b1282e44a39d659b87f120995e/users/f27665b1574a4db7b7a9771f7d775826/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:06:35.585296 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a72079-243a-4d98-931d-f2e545195681 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:35.720693 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a72079-243a-4d98-931d-f2e545195681 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:35.729924 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:35.760577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc748e4-31bf-4aaf-8408-87a44cdab5f5 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:35.769879 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461ccbd3-d599-4f49-9d6e-14d667a021a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:35.771046 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ccbd3-d599-4f49-9d6e-14d667a021a7 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:35.771334 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ccbd3-d599-4f49-9d6e-14d667a021a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:35.771608 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ccbd3-d599-4f49-9d6e-14d667a021a7 None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:35.772683 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461ccbd3-d599-4f49-9d6e-14d667a021a7 None admin] RBAC: Adding query filter params (name=network) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:35.773149 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461ccbd3-d599-4f49-9d6e-14d667a021a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:35.774997 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461ccbd3-d599-4f49-9d6e-14d667a021a7 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:35.782383 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 375/772] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Aug 31 05:06:35 2022] GET /identity/v3/services?name=network => generated 124 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:35.786923 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc08ed-269f-4ab6-9f75-4fea4d58d102 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:35.787203 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc08ed-269f-4ab6-9f75-4fea4d58d102 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:35.787537 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc08ed-269f-4ab6-9f75-4fea4d58d102 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:35.789007 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 376/773] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:35.789764 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abc08ed-269f-4ab6-9f75-4fea4d58d102 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:35.812471 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41781446-5f1f-4df0-b767-e619dbb2944b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:35.813139 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41781446-5f1f-4df0-b767-e619dbb2944b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:35.813414 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41781446-5f1f-4df0-b767-e619dbb2944b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:35.813684 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41781446-5f1f-4df0-b767-e619dbb2944b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:35.814301 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41781446-5f1f-4df0-b767-e619dbb2944b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:35.815699 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41781446-5f1f-4df0-b767-e619dbb2944b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:35.830447 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-41781446-5f1f-4df0-b767-e619dbb2944b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 05:06:35.830447 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:35.830447 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 05:06:35.830447 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 05:06:35.830447 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 05:06:35.830447 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 05:06:35.830447 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 05:06:35.830447 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:06:35.830447 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 05:06:35.830447 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:35.830447 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 05:06:35.830447 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:35.830447 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:35.830447 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:35.830447 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:35.830447 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:35.830447 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:35.832609 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:35.832609 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:35.832609 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:35.832609 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:35.832609 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:35.832609 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:35.832609 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 05:06:35.832609 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 05:06:35.832609 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 05:06:35.832609 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 05:06:35.832609 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:35.832609 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:35.832609 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:35.832609 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:35.832609 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:35.832609 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:35.834497 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:35.834497 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:35.834497 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:35.834497 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:35.834497 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:35.834497 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:35.834497 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:35.834497 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:35.834497 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:35.834497 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:35.834497 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 05:06:35.834497 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:06:35.834497 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 05:06:35.834497 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:35.834497 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 398/774] 10.222.0.36 () {56 vars in 1129 bytes} [Wed Aug 31 05:06:35 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:35.836126 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb854b-f082-4adb-b5e3-0ec41b2ad760 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:35.836126 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb854b-f082-4adb-b5e3-0ec41b2ad760 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:35.836126 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb854b-f082-4adb-b5e3-0ec41b2ad760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:35.851307 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-75eb854b-f082-4adb-b5e3-0ec41b2ad760 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:35.852549 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-75eb854b-f082-4adb-b5e3-0ec41b2ad760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:35.870942 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:35.923219 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abc08ed-269f-4ab6-9f75-4fea4d58d102 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:35.948162 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 399/775] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:06:35.949264 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eb854b-f082-4adb-b5e3-0ec41b2ad760 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:35.975442 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a18a89b-b4d2-412f-b83a-10b948c6ff6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:35.976109 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a18a89b-b4d2-412f-b83a-10b948c6ff6b None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:35.976385 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a18a89b-b4d2-412f-b83a-10b948c6ff6b None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:35.976651 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a18a89b-b4d2-412f-b83a-10b948c6ff6b None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:35.977490 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a18a89b-b4d2-412f-b83a-10b948c6ff6b None admin] RBAC: Adding query filter params (type=network) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:35.977786 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a18a89b-b4d2-412f-b83a-10b948c6ff6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:35.979337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a18a89b-b4d2-412f-b83a-10b948c6ff6b None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:35.993851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 377/776] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Aug 31 05:06:35 2022] GET /identity/v3/services?type=network => generated 350 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:35.999956 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a18a89b-b4d2-412f-b83a-10b948c6ff6b None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:36.063647 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eb854b-f082-4adb-b5e3-0ec41b2ad760 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:36.067379 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:36.104515 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56ccf3-c174-4a67-b192-57751d9a8eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:36.105416 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56ccf3-c174-4a67-b192-57751d9a8eb7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:36.105837 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56ccf3-c174-4a67-b192-57751d9a8eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:36.106290 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56ccf3-c174-4a67-b192-57751d9a8eb7 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:36.107160 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56ccf3-c174-4a67-b192-57751d9a8eb7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:36.107613 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56ccf3-c174-4a67-b192-57751d9a8eb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:36.109266 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56ccf3-c174-4a67-b192-57751d9a8eb7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:36.114723 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 400/777] 10.222.0.36 () {56 vars in 1124 bytes} [Wed Aug 31 05:06:35 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:36.126063 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56ccf3-c174-4a67-b192-57751d9a8eb7 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:36.151210 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:36.204883 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a18a89b-b4d2-412f-b83a-10b948c6ff6b None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:36.264879 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce0a885-d743-44df-8a60-8f0213912f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:36.265741 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce0a885-d743-44df-8a60-8f0213912f53 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:36.266337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce0a885-d743-44df-8a60-8f0213912f53 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:36.267868 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce0a885-d743-44df-8a60-8f0213912f53 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:36.268722 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce0a885-d743-44df-8a60-8f0213912f53 None admin] RBAC: Adding query filter params (interface=public, service_id=be1e9b079f574e989517207e23268caf, region_id=RegionOne) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:36.269352 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce0a885-d743-44df-8a60-8f0213912f53 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:36.271703 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce0a885-d743-44df-8a60-8f0213912f53 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:36.284573 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 378/778] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Aug 31 05:06:35 2022] GET /identity/v3/endpoints?service_id=be1e9b079f574e989517207e23268caf&interface=public®ion_id=RegionOne => generated 194 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:06:36.289980 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce0a885-d743-44df-8a60-8f0213912f53 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:36.297487 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56ccf3-c174-4a67-b192-57751d9a8eb7 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:36.340794 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7d6fe8-70d2-497b-aa7a-ce747b3b7dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:36.341681 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d6fe8-70d2-497b-aa7a-ce747b3b7dd2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:36.342402 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d6fe8-70d2-497b-aa7a-ce747b3b7dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:36.342814 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d6fe8-70d2-497b-aa7a-ce747b3b7dd2 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:36.343837 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7d6fe8-70d2-497b-aa7a-ce747b3b7dd2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:36.344690 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7d6fe8-70d2-497b-aa7a-ce747b3b7dd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:36.348557 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7d6fe8-70d2-497b-aa7a-ce747b3b7dd2 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:36.355346 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 401/779] 10.222.0.36 () {56 vars in 1124 bytes} [Wed Aug 31 05:06:36 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:06:36.361844 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7d6fe8-70d2-497b-aa7a-ce747b3b7dd2 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:36.418983 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:36.424228 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce0a885-d743-44df-8a60-8f0213912f53 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:36.448917 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:36.462271 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb75e31c-e65e-4623-8391-6a1fbfca0fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:36.463276 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75e31c-e65e-4623-8391-6a1fbfca0fb7 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:36.463698 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75e31c-e65e-4623-8391-6a1fbfca0fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:36.464105 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75e31c-e65e-4623-8391-6a1fbfca0fb7 None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:36.464851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb75e31c-e65e-4623-8391-6a1fbfca0fb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:36.466739 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb75e31c-e65e-4623-8391-6a1fbfca0fb7 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:36.472254 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7d6fe8-70d2-497b-aa7a-ce747b3b7dd2 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:36.472651 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 379/780] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Aug 31 05:06:36 2022] GET /identity/v3/services => generated 772 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:36.473564 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb75e31c-e65e-4623-8391-6a1fbfca0fb7 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:36.554137 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87518e89-e83d-4f7c-9114-43904bcc6209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:36.554952 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87518e89-e83d-4f7c-9114-43904bcc6209 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:36.555324 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87518e89-e83d-4f7c-9114-43904bcc6209 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:36.555686 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87518e89-e83d-4f7c-9114-43904bcc6209 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:36.576920 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87518e89-e83d-4f7c-9114-43904bcc6209 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:06:36.582855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-87518e89-e83d-4f7c-9114-43904bcc6209 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 05:06:36.582855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:36.582855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:36.582855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:36.582855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:36.582855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:36.582855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:36.582855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:36.582855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:36.582855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:36.582855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:36.582855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:36.582855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:06:36.582855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:06:36.582855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:06:36.582855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:06:36.583895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:06:36.583895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:06:36.583895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:06:36.583895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:06:36.583895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:36.583895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:36.583895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:36.583895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:36.583895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:36.583895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:36.583895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:36.583895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:36.583895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:36.583895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:36.583895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:36.583895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:36.584754 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:36.584754 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:36.584754 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:36.584754 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:36.584754 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:06:36.584754 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:06:36.584754 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:06:36.584754 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:06:36.584754 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:06:36.584754 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:06:36.584754 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 05:06:36.584754 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:36.590599 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 402/781] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 05:06:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:36.599227 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87518e89-e83d-4f7c-9114-43904bcc6209 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:36.627510 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb75e31c-e65e-4623-8391-6a1fbfca0fb7 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:36.666705 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f502da-6f7c-44a7-a896-381183ae5962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:36.667707 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f502da-6f7c-44a7-a896-381183ae5962 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:36.668192 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f502da-6f7c-44a7-a896-381183ae5962 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:36.668756 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f502da-6f7c-44a7-a896-381183ae5962 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:36.669733 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f502da-6f7c-44a7-a896-381183ae5962 None admin] RBAC: Adding query filter params (interface=public, service_id=b377683aaceb4191908884f0f39b0c37, region_id=RegionOne) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:36.670373 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f502da-6f7c-44a7-a896-381183ae5962 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:36.672008 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f502da-6f7c-44a7-a896-381183ae5962 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:36.678532 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 380/782] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Aug 31 05:06:36 2022] GET /identity/v3/endpoints?service_id=b377683aaceb4191908884f0f39b0c37&interface=public®ion_id=RegionOne => generated 194 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:36.684410 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f502da-6f7c-44a7-a896-381183ae5962 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:36.714680 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:36.769740 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87518e89-e83d-4f7c-9114-43904bcc6209 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:36.792967 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f502da-6f7c-44a7-a896-381183ae5962 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:36.808137 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e335e-f40b-4c72-a09c-a327989083e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:36.808746 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e335e-f40b-4c72-a09c-a327989083e5 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:36.809022 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e335e-f40b-4c72-a09c-a327989083e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:36.809287 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e335e-f40b-4c72-a09c-a327989083e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:36.810113 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9e335e-f40b-4c72-a09c-a327989083e5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:36.810373 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9e335e-f40b-4c72-a09c-a327989083e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:36.811931 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9e335e-f40b-4c72-a09c-a327989083e5 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:36.817845 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:36.822135 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 403/783] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Aug 31 05:06:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:06:36.828012 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e335e-f40b-4c72-a09c-a327989083e5 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:36.831053 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccce500-dda6-4d63-882e-549dcfc6ba43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:36.831678 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccce500-dda6-4d63-882e-549dcfc6ba43 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:36.832009 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccce500-dda6-4d63-882e-549dcfc6ba43 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:36.832201 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccce500-dda6-4d63-882e-549dcfc6ba43 None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:36.832816 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccce500-dda6-4d63-882e-549dcfc6ba43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:36.834223 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccce500-dda6-4d63-882e-549dcfc6ba43 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:36.840324 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 381/784] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Aug 31 05:06:36 2022] GET /identity/v3/services => generated 772 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:06:36.973194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e335e-f40b-4c72-a09c-a327989083e5 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:37.012209 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11b9775-b403-4b54-8e97-922a60c3594e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:37.012883 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b9775-b403-4b54-8e97-922a60c3594e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:37.013257 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b9775-b403-4b54-8e97-922a60c3594e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:37.013527 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b9775-b403-4b54-8e97-922a60c3594e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:37.026305 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11b9775-b403-4b54-8e97-922a60c3594e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:37.027767 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11b9775-b403-4b54-8e97-922a60c3594e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:37.042444 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-a11b9775-b403-4b54-8e97-922a60c3594e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:37.042444 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:37.042444 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:37.042444 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:37.042444 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:37.042444 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:37.042444 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:37.042444 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:37.042444 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:37.042444 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:37.042444 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:37.042444 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:37.042444 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:37.042444 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:37.042444 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:37.042444 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:37.043449 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:37.043449 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:37.043449 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:37.043449 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:37.043449 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:37.043449 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:37.043449 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:37.043449 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:37.043449 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:37.043449 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:37.043449 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:37.043449 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:37.043449 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:37.043449 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:37.043449 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:37.043449 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:37.043449 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:37.044430 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:37.044430 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:37.044430 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:37.044430 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 404/785] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:37.046616 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f5bed-920b-446a-839e-b5d878a34b28 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:37.046898 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f5bed-920b-446a-839e-b5d878a34b28 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:37.047165 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f5bed-920b-446a-839e-b5d878a34b28 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:37.049314 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11b9775-b403-4b54-8e97-922a60c3594e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:37.050086 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 382/786] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:37.054885 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3196081-6552-4fba-b97d-8002042c6e1a None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:37.055168 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3196081-6552-4fba-b97d-8002042c6e1a None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:37.055436 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3196081-6552-4fba-b97d-8002042c6e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:37.073266 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-d3196081-6552-4fba-b97d-8002042c6e1a None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:37.075355 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-d3196081-6552-4fba-b97d-8002042c6e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:37.106532 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:37.159395 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 383/787] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:06:37.163704 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11b9775-b403-4b54-8e97-922a60c3594e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:37.204618 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba63371-6478-410d-8684-60278a801bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:37.205605 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba63371-6478-410d-8684-60278a801bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:37.206109 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba63371-6478-410d-8684-60278a801bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:37.206594 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba63371-6478-410d-8684-60278a801bc5 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:37.207439 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba63371-6478-410d-8684-60278a801bc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:37.207951 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba63371-6478-410d-8684-60278a801bc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:37.209526 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba63371-6478-410d-8684-60278a801bc5 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:37.226442 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 405/788] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:06:37.227829 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3196081-6552-4fba-b97d-8002042c6e1a None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:37.257956 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:37.352462 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3196081-6552-4fba-b97d-8002042c6e1a None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:37.387457 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c4d69f-c5d2-445d-8d40-2a1541794aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:37.388122 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4d69f-c5d2-445d-8d40-2a1541794aab None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:37.388397 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4d69f-c5d2-445d-8d40-2a1541794aab None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:37.388666 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4d69f-c5d2-445d-8d40-2a1541794aab None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:37.395994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c4d69f-c5d2-445d-8d40-2a1541794aab None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:37.397470 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c4d69f-c5d2-445d-8d40-2a1541794aab None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:37.409194 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-07c4d69f-c5d2-445d-8d40-2a1541794aab None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 05:06:37.409194 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:37.409194 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:37.409194 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:37.409194 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:37.409194 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:37.409194 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:37.409194 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:37.409194 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:37.409194 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:37.409194 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:37.409194 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:37.409194 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 05:06:37.409194 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 05:06:37.409194 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 05:06:37.409194 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 05:06:37.410222 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:37.410222 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:37.410222 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 05:06:37.410222 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:37.410222 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 05:06:37.410222 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:37.410222 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:37.410222 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:37.410222 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:37.410222 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:37.410222 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:37.410222 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:37.410222 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:37.410222 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:37.410222 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:37.410222 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:37.410222 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:37.410222 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:37.411243 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:37.411243 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:37.411243 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 05:06:37.411243 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 05:06:37.411243 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 05:06:37.411243 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 05:06:37.411243 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 05:06:37.411243 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 05:06:37.411243 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 05:06:37.411243 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:37.411243 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 384/789] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Aug 31 05:06:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:37.418039 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba63371-6478-410d-8684-60278a801bc5 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:37.493715 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c4d69f-c5d2-445d-8d40-2a1541794aab None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:37.605726 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba63371-6478-410d-8684-60278a801bc5 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:37.642761 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:37.644363 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:37.685851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c4d69f-c5d2-445d-8d40-2a1541794aab None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:37.686689 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7263eb-de31-4697-8ea8-0922a69d3159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:37.687308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7263eb-de31-4697-8ea8-0922a69d3159 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:37.687577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7263eb-de31-4697-8ea8-0922a69d3159 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:37.687837 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7263eb-de31-4697-8ea8-0922a69d3159 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:37.688577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7263eb-de31-4697-8ea8-0922a69d3159 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:37.688875 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7263eb-de31-4697-8ea8-0922a69d3159 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:37.690332 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7263eb-de31-4697-8ea8-0922a69d3159 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:37.738334 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 406/790] 10.222.0.36 () {56 vars in 1149 bytes} [Wed Aug 31 05:06:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:37.742361 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7263eb-de31-4697-8ea8-0922a69d3159 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:37.763688 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b4f8df-6af6-4d61-a080-1692b28e25ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:37.764247 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4f8df-6af6-4d61-a080-1692b28e25ae None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:37.764573 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4f8df-6af6-4d61-a080-1692b28e25ae None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:37.764844 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4f8df-6af6-4d61-a080-1692b28e25ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:37.787900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b4f8df-6af6-4d61-a080-1692b28e25ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:06:37.793503 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-12b4f8df-6af6-4d61-a080-1692b28e25ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:06:37.793503 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:37.793503 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:37.793503 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:37.793503 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:37.793503 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:37.793503 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:37.793503 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:37.793503 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:37.793503 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:37.793503 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:37.793503 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:37.793503 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:06:37.793503 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:06:37.793503 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:06:37.793503 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:06:37.794508 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:06:37.794508 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:06:37.794508 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:06:37.794508 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:06:37.794508 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:37.794508 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:37.794508 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:37.794508 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:37.794508 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:37.794508 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:37.794508 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:37.794508 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:37.794508 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:37.794508 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:37.794508 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:37.794508 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:37.795367 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:37.795367 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:37.795367 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:37.795367 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:37.795367 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:06:37.795367 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:06:37.795367 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:06:37.795367 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:06:37.795367 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:06:37.795367 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:06:37.795367 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:06:37.795367 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:37.797939 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 385/791] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 05:06:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:06:37.806933 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b4f8df-6af6-4d61-a080-1692b28e25ae None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:37.911361 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7263eb-de31-4697-8ea8-0922a69d3159 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:37.977882 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa99e42-b5f9-4a71-aba6-90eb46883426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:37.979118 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa99e42-b5f9-4a71-aba6-90eb46883426 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:37.980354 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa99e42-b5f9-4a71-aba6-90eb46883426 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:37.981457 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa99e42-b5f9-4a71-aba6-90eb46883426 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:37.995637 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b4f8df-6af6-4d61-a080-1692b28e25ae None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:37.998868 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa99e42-b5f9-4a71-aba6-90eb46883426 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:38.001068 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa99e42-b5f9-4a71-aba6-90eb46883426 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:38.019474 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-2aa99e42-b5f9-4a71-aba6-90eb46883426 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:38.019474 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:38.019474 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:38.019474 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:38.019474 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:38.019474 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:38.019474 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:38.019474 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:38.019474 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:38.019474 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:38.019474 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:38.019474 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:38.019474 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:38.019474 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:38.019474 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:38.019474 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:38.020490 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:38.020490 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:38.020490 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:38.020490 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:38.020490 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:38.020490 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:38.020490 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:38.020490 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:38.020490 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:38.020490 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:38.020490 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:38.020490 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:38.020490 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:38.020490 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:38.020490 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:38.020490 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:38.020490 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:38.021410 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:38.021410 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:38.021410 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:38.023577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 407/792] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:38.029514 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02baf4bd-64a9-479f-bcbe-65f3cecb40f2 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:38.030174 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02baf4bd-64a9-479f-bcbe-65f3cecb40f2 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:38.030556 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02baf4bd-64a9-479f-bcbe-65f3cecb40f2 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:38.036059 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 408/793] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:38 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:38.036059 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02baf4bd-64a9-479f-bcbe-65f3cecb40f2 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:38.037123 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edec8f21-e535-4060-aca0-e2036fd3aba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:38.037763 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec8f21-e535-4060-aca0-e2036fd3aba6 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:38.038051 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec8f21-e535-4060-aca0-e2036fd3aba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:38.038327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec8f21-e535-4060-aca0-e2036fd3aba6 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:38.039003 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edec8f21-e535-4060-aca0-e2036fd3aba6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:38.039302 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edec8f21-e535-4060-aca0-e2036fd3aba6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:38.041857 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edec8f21-e535-4060-aca0-e2036fd3aba6 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:38.052588 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 386/794] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:06:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:38.054446 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8afe05-3ae0-42cc-80e2-b14f534a772e None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:38.054735 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8afe05-3ae0-42cc-80e2-b14f534a772e None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:38.055006 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8afe05-3ae0-42cc-80e2-b14f534a772e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:38.062876 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:38.068862 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:38.083844 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-fc8afe05-3ae0-42cc-80e2-b14f534a772e None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:38.085082 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-fc8afe05-3ae0-42cc-80e2-b14f534a772e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:38.162031 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 387/795] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:06:38.162031 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8afe05-3ae0-42cc-80e2-b14f534a772e None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:38.200891 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02baf4bd-64a9-479f-bcbe-65f3cecb40f2 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:38.264313 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cafdf1-03db-49d1-a18f-746e2b4335ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:38.264920 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cafdf1-03db-49d1-a18f-746e2b4335ce None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:38.265193 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cafdf1-03db-49d1-a18f-746e2b4335ce None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:38.265459 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cafdf1-03db-49d1-a18f-746e2b4335ce None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:38.266126 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cafdf1-03db-49d1-a18f-746e2b4335ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:38.266476 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cafdf1-03db-49d1-a18f-746e2b4335ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:38.268608 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cafdf1-03db-49d1-a18f-746e2b4335ce None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:38.294035 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 409/796] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:38.299656 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cafdf1-03db-49d1-a18f-746e2b4335ce None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:38.342296 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8afe05-3ae0-42cc-80e2-b14f534a772e None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:38.380026 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:38.394978 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fef41d0-1e2e-472c-855b-1d50afbf6fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:38.396041 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef41d0-1e2e-472c-855b-1d50afbf6fee None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:38.396568 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef41d0-1e2e-472c-855b-1d50afbf6fee None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:38.397029 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef41d0-1e2e-472c-855b-1d50afbf6fee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:38.417684 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fef41d0-1e2e-472c-855b-1d50afbf6fee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:38.421417 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fef41d0-1e2e-472c-855b-1d50afbf6fee None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:38.454337 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:38.463070 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-9fef41d0-1e2e-472c-855b-1d50afbf6fee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:38.463070 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:38.463070 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:38.463070 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:38.463070 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:38.463070 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:38.463070 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:38.463070 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:38.463070 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:38.463070 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:38.463070 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:38.463070 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:38.463070 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:38.463070 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:38.463070 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:38.463070 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:38.464102 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:38.464102 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:38.464102 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:38.464102 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:38.464102 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:38.464102 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:38.464102 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:38.464102 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:38.464102 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:38.464102 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:38.464102 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:38.464102 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:38.464102 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:38.464102 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:38.464102 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:38.464102 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:38.464102 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:38.465007 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:38.465007 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:38.465007 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:38.465197 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 388/797] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:38.471804 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fef41d0-1e2e-472c-855b-1d50afbf6fee None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:38.474340 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cafdf1-03db-49d1-a18f-746e2b4335ce None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:38.525090 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b4af2-28d9-4291-818f-fc3e3b28892f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:38.525755 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b4af2-28d9-4291-818f-fc3e3b28892f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:38.526069 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b4af2-28d9-4291-818f-fc3e3b28892f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:38.526345 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b4af2-28d9-4291-818f-fc3e3b28892f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:38.534225 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371b4af2-28d9-4291-818f-fc3e3b28892f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:38.535716 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371b4af2-28d9-4291-818f-fc3e3b28892f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:38.548646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-371b4af2-28d9-4291-818f-fc3e3b28892f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:38.548646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:38.548646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:38.548646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:38.548646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:38.548646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:38.548646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:38.548646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:38.548646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:38.548646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:38.548646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:38.548646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:38.548646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 05:06:38.548646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 05:06:38.548646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 05:06:38.549627 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 05:06:38.549627 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:38.549627 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:38.549627 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:38.549627 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:38.549627 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:38.549627 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:38.549627 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:38.549627 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:38.549627 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:38.549627 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:38.549627 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:38.549627 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:38.549627 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:38.549627 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:38.549627 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:38.549627 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:38.550560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 05:06:38.550560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 05:06:38.550560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 05:06:38.550560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 05:06:38.550560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 05:06:38.550560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 05:06:38.550560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:38.550560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:38.550560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 410/798] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:06:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:06:38.551124 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b4af2-28d9-4291-818f-fc3e3b28892f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:38.659200 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b4af2-28d9-4291-818f-fc3e3b28892f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:38.697638 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5211f824-c753-445c-8723-74cbbe12bb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:38.698318 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211f824-c753-445c-8723-74cbbe12bb63 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:38.698595 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211f824-c753-445c-8723-74cbbe12bb63 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:38.698859 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211f824-c753-445c-8723-74cbbe12bb63 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:38.711972 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5211f824-c753-445c-8723-74cbbe12bb63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:06:38.719065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-5211f824-c753-445c-8723-74cbbe12bb63 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:06:38.719065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:38.719065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:38.719065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:38.719065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:38.719065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:38.719065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:38.719065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:38.719065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:38.719065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:38.719065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:38.719065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:38.719065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:06:38.719065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:06:38.719065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:06:38.719065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:06:38.720041 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:06:38.720041 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:06:38.720041 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:06:38.720041 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:06:38.720041 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:38.720041 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:38.720041 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:38.720041 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:38.720041 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:38.720041 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:38.720041 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:38.720041 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:38.720041 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:38.720041 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:38.720041 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:38.720041 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:38.720887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:38.720887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:38.720887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:38.720887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:38.720887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:06:38.720887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:06:38.720887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:06:38.720887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:06:38.720887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:06:38.720887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:06:38.720887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:06:38.720887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:38.725172 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 411/799] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 05:06:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:38.726199 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5211f824-c753-445c-8723-74cbbe12bb63 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:38.729864 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:38.735760 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fef41d0-1e2e-472c-855b-1d50afbf6fee None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:38.774205 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8833f197-f6ba-4bc0-8dbd-81a346981d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:38.775223 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833f197-f6ba-4bc0-8dbd-81a346981d64 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:38.775713 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833f197-f6ba-4bc0-8dbd-81a346981d64 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:38.776200 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833f197-f6ba-4bc0-8dbd-81a346981d64 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:38.777310 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8833f197-f6ba-4bc0-8dbd-81a346981d64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:38.777803 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8833f197-f6ba-4bc0-8dbd-81a346981d64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:38.779517 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8833f197-f6ba-4bc0-8dbd-81a346981d64 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:38.792711 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 389/800] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:38.793931 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8833f197-f6ba-4bc0-8dbd-81a346981d64 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:38.832348 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5211f824-c753-445c-8723-74cbbe12bb63 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:38.842179 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:38.867013 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d98c80-ecff-44d4-8fcd-c6e23d517901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:38.867622 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d98c80-ecff-44d4-8fcd-c6e23d517901 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:38.867894 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d98c80-ecff-44d4-8fcd-c6e23d517901 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:38.868473 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d98c80-ecff-44d4-8fcd-c6e23d517901 None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:38.869150 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d98c80-ecff-44d4-8fcd-c6e23d517901 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:38.869454 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d98c80-ecff-44d4-8fcd-c6e23d517901 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:38.870905 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d98c80-ecff-44d4-8fcd-c6e23d517901 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:38.883218 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 412/801] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 05:06:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:38.884005 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d98c80-ecff-44d4-8fcd-c6e23d517901 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:38.908551 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8833f197-f6ba-4bc0-8dbd-81a346981d64 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:38.942715 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4d2970-2a49-4733-ab2a-188281ae69af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:38.943311 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d2970-2a49-4733-ab2a-188281ae69af None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:38.943581 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d2970-2a49-4733-ab2a-188281ae69af None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:38.943843 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d2970-2a49-4733-ab2a-188281ae69af None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:38.944487 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4d2970-2a49-4733-ab2a-188281ae69af None admin] RBAC: Adding query filter params (name=admin) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:38.944782 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4d2970-2a49-4733-ab2a-188281ae69af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:38.946172 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4d2970-2a49-4733-ab2a-188281ae69af None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:38.953378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:38.959999 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 390/802] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:06:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:38.959999 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120319a-2dae-46a7-b0d1-ffda9bd74fe8 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:38.959999 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120319a-2dae-46a7-b0d1-ffda9bd74fe8 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:38.959999 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120319a-2dae-46a7-b0d1-ffda9bd74fe8 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:38.959999 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 391/803] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:38.959999 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a120319a-2dae-46a7-b0d1-ffda9bd74fe8 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:39.001816 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d98c80-ecff-44d4-8fcd-c6e23d517901 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:39.036389 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a7684-09a3-4354-9423-d9039d64e516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:39.037063 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a7684-09a3-4354-9423-d9039d64e516 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:39.037349 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a7684-09a3-4354-9423-d9039d64e516 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:39.037618 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a7684-09a3-4354-9423-d9039d64e516 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:39.044752 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927a7684-09a3-4354-9423-d9039d64e516 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:39.046227 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927a7684-09a3-4354-9423-d9039d64e516 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:39.058729 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-927a7684-09a3-4354-9423-d9039d64e516 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 05:06:39.058729 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:39.058729 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:39.058729 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:39.058729 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:39.058729 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:39.058729 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:39.058729 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:39.058729 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:39.058729 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:39.058729 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:39.058729 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:39.058729 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 05:06:39.058729 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 05:06:39.058729 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 05:06:39.058729 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 05:06:39.059879 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:39.059879 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:39.059879 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 05:06:39.059879 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:39.059879 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 05:06:39.059879 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:39.059879 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:39.059879 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:39.059879 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:39.059879 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:39.059879 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:39.059879 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:39.059879 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:39.059879 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:39.059879 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:39.059879 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:39.059879 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:39.059879 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:39.061000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:39.061000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:39.061000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 05:06:39.061000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 05:06:39.061000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 05:06:39.061000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 05:06:39.061000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 05:06:39.061000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 05:06:39.061000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 05:06:39.061000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:39.061000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 413/804] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 05:06:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:39.061000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a7684-09a3-4354-9423-d9039d64e516 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:39.063191 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a120319a-2dae-46a7-b0d1-ffda9bd74fe8 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:39.071503 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:39.103870 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f5506c-e29c-42a3-b8eb-f51652480e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:39.104523 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5506c-e29c-42a3-b8eb-f51652480e1a None admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:39.104649 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5506c-e29c-42a3-b8eb-f51652480e1a None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:39.104947 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5506c-e29c-42a3-b8eb-f51652480e1a None admin] PATH_INFO: `/v3/projects/e01ce1b1282e44a39d659b87f120995e/users/09bf6db88ce8494a937211a814c9ff84/roles/207bf98e31774cc0b21249de6d96155b` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:39.122873 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f5506c-e29c-42a3-b8eb-f51652480e1a None admin] RBAC: Authorizing `identity:create_grant(project_id=e01ce1b1282e44a39d659b87f120995e, user_id=09bf6db88ce8494a937211a814c9ff84, role_id=207bf98e31774cc0b21249de6d96155b)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:39.124375 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f5506c-e29c-42a3-b8eb-f51652480e1a None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:39.139054 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 392/805] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 05:06:38 2022] PUT /identity/v3/projects/e01ce1b1282e44a39d659b87f120995e/users/09bf6db88ce8494a937211a814c9ff84/roles/207bf98e31774cc0b21249de6d96155b => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:06:39.140407 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172dcfd-cc6a-4708-a44e-bfaba5e68c3b None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:39.140661 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172dcfd-cc6a-4708-a44e-bfaba5e68c3b None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:39.140913 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172dcfd-cc6a-4708-a44e-bfaba5e68c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:39.161495 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-9172dcfd-cc6a-4708-a44e-bfaba5e68c3b None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:39.162787 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-9172dcfd-cc6a-4708-a44e-bfaba5e68c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:39.245964 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 393/806] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:06:39.245964 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6323245-414f-4748-96eb-33acf2a337f7 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:39.245964 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6323245-414f-4748-96eb-33acf2a337f7 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:39.245964 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6323245-414f-4748-96eb-33acf2a337f7 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:39.245964 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 394/807] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:39.245964 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6323245-414f-4748-96eb-33acf2a337f7 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:39.308628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a7684-09a3-4354-9423-d9039d64e516 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:39.340195 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:39.355288 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:39.355536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22bf345-6206-458b-9154-9fb6aefa52a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:39.356807 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22bf345-6206-458b-9154-9fb6aefa52a0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:39.357201 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22bf345-6206-458b-9154-9fb6aefa52a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:39.357593 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22bf345-6206-458b-9154-9fb6aefa52a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:39.373777 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22bf345-6206-458b-9154-9fb6aefa52a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:39.375265 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6323245-414f-4748-96eb-33acf2a337f7 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:39.375578 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22bf345-6206-458b-9154-9fb6aefa52a0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:39.395146 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-a22bf345-6206-458b-9154-9fb6aefa52a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:39.395146 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:39.395146 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:39.395146 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:39.395146 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:39.395146 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:39.395146 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:39.395146 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:39.395146 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:39.395146 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:39.395146 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:39.395146 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:39.395146 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:39.395146 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:39.395146 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:39.395146 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:39.396307 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:39.396307 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:39.396307 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:39.396307 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:39.396307 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:39.396307 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:39.396307 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:39.396307 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:39.396307 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:39.396307 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:39.396307 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:39.396307 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:39.396307 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:39.396307 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:39.396307 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:39.396307 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:39.396307 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:39.397398 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:39.397398 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:39.397398 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:39.398382 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 414/808] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:39.399685 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41539e5-8b3b-4dd7-afb5-4b26c6eb1a90 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:39.400081 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41539e5-8b3b-4dd7-afb5-4b26c6eb1a90 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:39.400420 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41539e5-8b3b-4dd7-afb5-4b26c6eb1a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:39.415434 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99d592-8369-4553-bb8d-6537a3361c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:39.416334 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99d592-8369-4553-bb8d-6537a3361c7a None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:39.416808 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99d592-8369-4553-bb8d-6537a3361c7a None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:39.417074 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99d592-8369-4553-bb8d-6537a3361c7a None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:39.417798 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a99d592-8369-4553-bb8d-6537a3361c7a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:39.418136 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a99d592-8369-4553-bb8d-6537a3361c7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:39.419745 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a99d592-8369-4553-bb8d-6537a3361c7a None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:39.424360 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b41539e5-8b3b-4dd7-afb5-4b26c6eb1a90 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:39.425978 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b41539e5-8b3b-4dd7-afb5-4b26c6eb1a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:39.438044 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 395/809] 10.222.0.36 () {56 vars in 1135 bytes} [Wed Aug 31 05:06:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:39.438894 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99d592-8369-4553-bb8d-6537a3361c7a None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:39.506286 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 415/810] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:06:39.507258 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41539e5-8b3b-4dd7-afb5-4b26c6eb1a90 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:39.541945 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99d592-8369-4553-bb8d-6537a3361c7a None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:39.581943 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6acad87-6815-4ccb-b68b-5a878e18526b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:39.582581 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6acad87-6815-4ccb-b68b-5a878e18526b None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:39.582851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6acad87-6815-4ccb-b68b-5a878e18526b None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:39.583115 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6acad87-6815-4ccb-b68b-5a878e18526b None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:39.584261 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6acad87-6815-4ccb-b68b-5a878e18526b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:39.584564 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6acad87-6815-4ccb-b68b-5a878e18526b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:39.586256 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6acad87-6815-4ccb-b68b-5a878e18526b None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:39.601897 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 396/811] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:39.607699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6acad87-6815-4ccb-b68b-5a878e18526b None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:39.611643 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:39.624280 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41539e5-8b3b-4dd7-afb5-4b26c6eb1a90 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:39.626771 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:39.659588 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2152cae-2730-4b28-b3e6-97efb8cb0cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:39.660223 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2152cae-2730-4b28-b3e6-97efb8cb0cf2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:39.660498 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2152cae-2730-4b28-b3e6-97efb8cb0cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:39.660761 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2152cae-2730-4b28-b3e6-97efb8cb0cf2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:39.671379 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2152cae-2730-4b28-b3e6-97efb8cb0cf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:39.672828 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2152cae-2730-4b28-b3e6-97efb8cb0cf2 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:39.687583 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-b2152cae-2730-4b28-b3e6-97efb8cb0cf2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:39.687583 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:39.687583 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:39.687583 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:39.687583 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:39.687583 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:39.687583 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:39.687583 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:39.687583 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:39.687583 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:39.687583 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:39.687583 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:39.687583 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:39.687583 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:39.687583 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:39.687583 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:39.688568 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:39.688568 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:39.688568 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:39.688568 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:39.688568 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:39.688568 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:39.688568 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:39.688568 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:39.688568 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:39.688568 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:39.688568 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:39.688568 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:39.688568 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:39.688568 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:39.688568 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:39.688568 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:39.688568 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:39.689463 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:39.689463 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:39.689463 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:39.693909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 416/812] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:39.694357 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2152cae-2730-4b28-b3e6-97efb8cb0cf2 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:39.728925 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6acad87-6815-4ccb-b68b-5a878e18526b None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:39.770701 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e0d402-a631-4be7-8243-a30ff2e35433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:39.771689 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e0d402-a631-4be7-8243-a30ff2e35433 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:39.772163 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e0d402-a631-4be7-8243-a30ff2e35433 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:39.772740 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e0d402-a631-4be7-8243-a30ff2e35433 None admin] PATH_INFO: `/v3/services/network` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:39.773536 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e0d402-a631-4be7-8243-a30ff2e35433 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:39.775287 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e0d402-a631-4be7-8243-a30ff2e35433 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:39.785114 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-c5e0d402-a631-4be7-8243-a30ff2e35433 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 05:06:39.785114 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:39.785114 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 05:06:39.785114 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 05:06:39.785114 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 05:06:39.785114 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 05:06:39.785114 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 05:06:39.785114 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:06:39.785114 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 05:06:39.785114 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:39.785114 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 05:06:39.785114 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:39.785114 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:39.785114 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:39.785114 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:39.785114 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:39.785114 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:39.786134 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:39.786134 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:39.786134 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:39.786134 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:39.786134 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:39.786134 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:39.786134 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 05:06:39.786134 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 05:06:39.786134 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 05:06:39.786134 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 05:06:39.786134 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:39.786134 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:39.786134 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:39.786134 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:39.786134 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:39.786134 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:39.786976 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:39.786976 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:39.786976 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:39.786976 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:39.786976 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:39.786976 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:39.786976 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:39.786976 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:39.786976 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:39.786976 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:39.786976 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 05:06:39.786976 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:06:39.786976 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 05:06:39.786976 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:39.789776 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 397/813] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:06:39 2022] GET /identity/v3/services/network => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:39.790783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e0d402-a631-4be7-8243-a30ff2e35433 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:39.800282 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2152cae-2730-4b28-b3e6-97efb8cb0cf2 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:39.834392 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c2995e-c2f2-4b05-b76d-4cf99709eea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:39.835022 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c2995e-c2f2-4b05-b76d-4cf99709eea4 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:39.835297 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c2995e-c2f2-4b05-b76d-4cf99709eea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:39.835561 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c2995e-c2f2-4b05-b76d-4cf99709eea4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:39.843616 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c2995e-c2f2-4b05-b76d-4cf99709eea4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:39.845218 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c2995e-c2f2-4b05-b76d-4cf99709eea4 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:39.858367 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:39.859280 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-82c2995e-c2f2-4b05-b76d-4cf99709eea4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 05:06:39.859280 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:39.859280 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:39.859280 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:39.859280 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:39.859280 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:39.859280 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:39.859280 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:39.859280 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:39.859280 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:39.859280 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:39.859280 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:39.859280 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 05:06:39.859280 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 05:06:39.859280 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 05:06:39.859280 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 05:06:39.860216 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:39.860216 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:39.860216 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 05:06:39.860216 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:39.860216 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 05:06:39.860216 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:39.860216 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:39.860216 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:39.860216 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:39.860216 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:39.860216 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:39.860216 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:39.860216 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:39.860216 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:39.860216 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:39.860216 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:39.860216 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:39.860216 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:39.861245 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:39.861245 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:39.861245 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 05:06:39.861245 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 05:06:39.861245 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 05:06:39.861245 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 05:06:39.861245 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 05:06:39.861245 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 05:06:39.861245 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 05:06:39.861245 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:39.861245 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 417/814] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Aug 31 05:06:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:39.861245 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c2995e-c2f2-4b05-b76d-4cf99709eea4 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:39.882288 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:39.899182 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e0d402-a631-4be7-8243-a30ff2e35433 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:39.935489 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd99334c-95e9-44a3-89ff-a149d42c1379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:39.936092 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99334c-95e9-44a3-89ff-a149d42c1379 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:39.936375 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99334c-95e9-44a3-89ff-a149d42c1379 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:39.936642 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99334c-95e9-44a3-89ff-a149d42c1379 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:39.937289 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd99334c-95e9-44a3-89ff-a149d42c1379 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:39.937588 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd99334c-95e9-44a3-89ff-a149d42c1379 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:39.939049 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd99334c-95e9-44a3-89ff-a149d42c1379 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:39.955644 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 398/815] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:39.956380 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd99334c-95e9-44a3-89ff-a149d42c1379 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:39.967572 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c2995e-c2f2-4b05-b76d-4cf99709eea4 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:40.002317 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e1e14-118f-47cf-b158-8ceeb6385417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:40.003053 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e1e14-118f-47cf-b158-8ceeb6385417 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:40.003222 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e1e14-118f-47cf-b158-8ceeb6385417 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:40.003494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e1e14-118f-47cf-b158-8ceeb6385417 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:40.004075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5e1e14-118f-47cf-b158-8ceeb6385417 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:40.005442 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5e1e14-118f-47cf-b158-8ceeb6385417 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:40.014848 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-7a5e1e14-118f-47cf-b158-8ceeb6385417 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 05:06:40.014848 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:40.014848 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 05:06:40.014848 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 05:06:40.014848 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 05:06:40.014848 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 05:06:40.014848 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 05:06:40.014848 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:06:40.014848 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 05:06:40.014848 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:40.014848 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 05:06:40.014848 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:40.014848 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:40.014848 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:40.014848 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:40.014848 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:40.014848 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:40.015829 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:40.015829 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:40.015829 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:40.015829 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:40.015829 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:40.015829 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:40.015829 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 05:06:40.015829 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 05:06:40.015829 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 05:06:40.015829 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 05:06:40.015829 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:40.015829 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:40.015829 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:40.015829 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:40.015829 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:40.015829 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:40.016671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:40.016671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:40.016671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:40.016671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:40.016671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:40.016671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:40.016671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:40.016671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:40.016671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:40.016671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:40.016671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 05:06:40.016671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:06:40.016671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 05:06:40.016671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:40.016671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 418/816] 10.222.0.36 () {56 vars in 1129 bytes} [Wed Aug 31 05:06:39 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:40.017479 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e1e14-118f-47cf-b158-8ceeb6385417 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:40.072017 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd99334c-95e9-44a3-89ff-a149d42c1379 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:40.097149 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:40.100793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:40.110225 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95af2406-6f41-4606-a178-024e532892ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:40.110806 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af2406-6f41-4606-a178-024e532892ac None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:40.111029 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af2406-6f41-4606-a178-024e532892ac None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:40.111295 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af2406-6f41-4606-a178-024e532892ac None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:40.111923 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95af2406-6f41-4606-a178-024e532892ac None admin] RBAC: Adding query filter params (name=network) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:40.112217 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95af2406-6f41-4606-a178-024e532892ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:40.113588 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95af2406-6f41-4606-a178-024e532892ac None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:40.118604 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 399/817] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Aug 31 05:06:39 2022] GET /identity/v3/services?name=network => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:40.119340 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e1e14-118f-47cf-b158-8ceeb6385417 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:40.123695 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95af2406-6f41-4606-a178-024e532892ac None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:40.155044 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1154bd9-cd79-44f8-8bb1-cc0d795a77da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:40.155821 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1154bd9-cd79-44f8-8bb1-cc0d795a77da None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:40.156135 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1154bd9-cd79-44f8-8bb1-cc0d795a77da None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:40.156401 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1154bd9-cd79-44f8-8bb1-cc0d795a77da None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:40.157164 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1154bd9-cd79-44f8-8bb1-cc0d795a77da None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:40.157483 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1154bd9-cd79-44f8-8bb1-cc0d795a77da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:40.158948 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1154bd9-cd79-44f8-8bb1-cc0d795a77da None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:40.177438 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 419/818] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Aug 31 05:06:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:40.178405 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1154bd9-cd79-44f8-8bb1-cc0d795a77da None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:40.229952 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95af2406-6f41-4606-a178-024e532892ac None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:40.265275 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f2d14e-aa95-403d-8cd4-72a8de8bda68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:40.265899 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2d14e-aa95-403d-8cd4-72a8de8bda68 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:40.266191 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2d14e-aa95-403d-8cd4-72a8de8bda68 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:40.266454 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2d14e-aa95-403d-8cd4-72a8de8bda68 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:40.273531 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f2d14e-aa95-403d-8cd4-72a8de8bda68 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:40.275059 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f2d14e-aa95-403d-8cd4-72a8de8bda68 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:40.284244 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1154bd9-cd79-44f8-8bb1-cc0d795a77da None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:40.290735 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-f6f2d14e-aa95-403d-8cd4-72a8de8bda68 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:40.290735 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:40.290735 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:40.290735 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:40.290735 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:40.290735 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:40.290735 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:40.290735 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:40.290735 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:40.290735 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:40.290735 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:40.290735 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:40.290735 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 05:06:40.290735 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 05:06:40.290735 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 05:06:40.291637 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 05:06:40.291637 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:40.291637 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:40.291637 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:40.291637 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:40.291637 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:40.291637 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:40.291637 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:40.291637 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:40.291637 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:40.291637 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:40.291637 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:40.291637 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:40.291637 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:40.291637 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:40.291637 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:40.291637 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:40.292519 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 05:06:40.292519 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 05:06:40.292519 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 05:06:40.292519 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 05:06:40.292519 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 05:06:40.292519 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 05:06:40.292519 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:40.292519 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:40.292519 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 400/819] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:06:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:40.293002 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f2d14e-aa95-403d-8cd4-72a8de8bda68 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:40.326048 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afca89f-60d1-44cb-a6e4-4bf92ad3073c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:40.326652 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afca89f-60d1-44cb-a6e4-4bf92ad3073c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:40.326924 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afca89f-60d1-44cb-a6e4-4bf92ad3073c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:40.327194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afca89f-60d1-44cb-a6e4-4bf92ad3073c None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:40.327845 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afca89f-60d1-44cb-a6e4-4bf92ad3073c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:40.328139 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afca89f-60d1-44cb-a6e4-4bf92ad3073c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:40.329518 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afca89f-60d1-44cb-a6e4-4bf92ad3073c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:40.334675 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 420/820] 10.222.0.36 () {56 vars in 1124 bytes} [Wed Aug 31 05:06:40 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:40.335347 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afca89f-60d1-44cb-a6e4-4bf92ad3073c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:40.337929 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:40.355331 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:40.406982 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f2d14e-aa95-403d-8cd4-72a8de8bda68 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:40.442335 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e226a3-4b09-448a-9e1f-855e4e33eca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:40.443023 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e226a3-4b09-448a-9e1f-855e4e33eca1 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:40.443359 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e226a3-4b09-448a-9e1f-855e4e33eca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:40.443713 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e226a3-4b09-448a-9e1f-855e4e33eca1 None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:40.444460 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e226a3-4b09-448a-9e1f-855e4e33eca1 None admin] RBAC: Adding query filter params (type=network) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:40.444815 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e226a3-4b09-448a-9e1f-855e4e33eca1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:40.446274 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e226a3-4b09-448a-9e1f-855e4e33eca1 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:40.449663 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afca89f-60d1-44cb-a6e4-4bf92ad3073c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:40.450956 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 401/821] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Aug 31 05:06:40 2022] GET /identity/v3/services?type=network => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:40.451708 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e226a3-4b09-448a-9e1f-855e4e33eca1 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:40.492292 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172410e1-da55-42b8-8fda-213b575ab938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:40.492906 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172410e1-da55-42b8-8fda-213b575ab938 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:40.493536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172410e1-da55-42b8-8fda-213b575ab938 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:40.493844 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172410e1-da55-42b8-8fda-213b575ab938 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:40.555274 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e226a3-4b09-448a-9e1f-855e4e33eca1 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:40.566315 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172410e1-da55-42b8-8fda-213b575ab938 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:40.567867 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172410e1-da55-42b8-8fda-213b575ab938 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:40.570030 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:40.571729 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:40.592947 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1135f7a7-2fad-41db-b45a-96186d47d086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:40.593816 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1135f7a7-2fad-41db-b45a-96186d47d086 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:40.594114 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1135f7a7-2fad-41db-b45a-96186d47d086 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:40.594528 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1135f7a7-2fad-41db-b45a-96186d47d086 None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:40.595518 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1135f7a7-2fad-41db-b45a-96186d47d086 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:40.595819 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1135f7a7-2fad-41db-b45a-96186d47d086 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:40.597385 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1135f7a7-2fad-41db-b45a-96186d47d086 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:40.609501 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 402/822] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 05:06:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:40.610366 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1135f7a7-2fad-41db-b45a-96186d47d086 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:40.694787 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 421/823] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:06:40 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 05:06:40.694787 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172410e1-da55-42b8-8fda-213b575ab938 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:40.724943 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1135f7a7-2fad-41db-b45a-96186d47d086 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:40.760536 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f3517-a8da-4794-bfe1-8d6745debfe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:40.761137 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f3517-a8da-4794-bfe1-8d6745debfe2 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:40.765506 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f3517-a8da-4794-bfe1-8d6745debfe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:40.765506 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f3517-a8da-4794-bfe1-8d6745debfe2 None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:40.765506 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94f3517-a8da-4794-bfe1-8d6745debfe2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:40.765506 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94f3517-a8da-4794-bfe1-8d6745debfe2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:40.765506 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94f3517-a8da-4794-bfe1-8d6745debfe2 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:40.771641 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 403/824] 10.222.0.36 () {56 vars in 1124 bytes} [Wed Aug 31 05:06:40 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:40.771988 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f3517-a8da-4794-bfe1-8d6745debfe2 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:40.800960 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172410e1-da55-42b8-8fda-213b575ab938 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:40.802848 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:40.812491 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:40.839879 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08909a14-316c-42d4-a815-da61389ecf74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:40.840494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08909a14-316c-42d4-a815-da61389ecf74 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:40.840770 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08909a14-316c-42d4-a815-da61389ecf74 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:40.841065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08909a14-316c-42d4-a815-da61389ecf74 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:40.841755 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08909a14-316c-42d4-a815-da61389ecf74 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:40.843143 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08909a14-316c-42d4-a815-da61389ecf74 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:40.865307 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 422/825] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:06:40 2022] POST /identity/v3/endpoints => generated 337 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:06:40.866606 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08909a14-316c-42d4-a815-da61389ecf74 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:40.876453 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f3517-a8da-4794-bfe1-8d6745debfe2 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:40.928527 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09afbdb-261e-4517-b3a4-22d35f7acbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:40.929224 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09afbdb-261e-4517-b3a4-22d35f7acbed None admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:40.929517 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09afbdb-261e-4517-b3a4-22d35f7acbed None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:40.929789 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09afbdb-261e-4517-b3a4-22d35f7acbed None admin] PATH_INFO: `/v3/projects/e01ce1b1282e44a39d659b87f120995e/users/d9b82080a73d4c4a8ec08b0e67d6d11c/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:40.932262 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09afbdb-261e-4517-b3a4-22d35f7acbed None admin] RBAC: Authorizing `identity:create_grant(project_id=e01ce1b1282e44a39d659b87f120995e, user_id=d9b82080a73d4c4a8ec08b0e67d6d11c, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:40.933608 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09afbdb-261e-4517-b3a4-22d35f7acbed None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:40.958197 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 404/826] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 05:06:40 2022] PUT /identity/v3/projects/e01ce1b1282e44a39d659b87f120995e/users/d9b82080a73d4c4a8ec08b0e67d6d11c/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:06:40.959765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09afbdb-261e-4517-b3a4-22d35f7acbed None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:41.017670 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08909a14-316c-42d4-a815-da61389ecf74 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:41.065960 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee0a7cb-4b2c-4c1c-8913-58f1bf2d8f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:41.067024 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0a7cb-4b2c-4c1c-8913-58f1bf2d8f9f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:41.067564 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0a7cb-4b2c-4c1c-8913-58f1bf2d8f9f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:41.068822 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0a7cb-4b2c-4c1c-8913-58f1bf2d8f9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:41.081894 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee0a7cb-4b2c-4c1c-8913-58f1bf2d8f9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:41.083934 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee0a7cb-4b2c-4c1c-8913-58f1bf2d8f9f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:41.102759 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-6ee0a7cb-4b2c-4c1c-8913-58f1bf2d8f9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:41.102759 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:41.102759 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:41.102759 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:41.102759 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:41.102759 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:41.102759 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:41.102759 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:41.102759 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:41.102759 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:41.102759 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:41.102759 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:41.102759 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:41.102759 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:41.102759 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:41.102759 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:41.103791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:41.103791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:41.103791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:41.103791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:41.103791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:41.103791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:41.103791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:41.103791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:41.103791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:41.103791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:41.103791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:41.103791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:41.103791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:41.103791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:41.103791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:41.103791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:41.103791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:41.104695 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:41.104695 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:41.104695 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:41.107122 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 423/827] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:41.116360 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee0a7cb-4b2c-4c1c-8913-58f1bf2d8f9f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:41.119178 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:41.124547 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09afbdb-261e-4517-b3a4-22d35f7acbed None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:41.138087 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:41.169226 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c10a3d5-f659-4cee-a979-8ce2f2cb1738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:41.169827 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10a3d5-f659-4cee-a979-8ce2f2cb1738 None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:41.170130 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10a3d5-f659-4cee-a979-8ce2f2cb1738 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:41.170423 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10a3d5-f659-4cee-a979-8ce2f2cb1738 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:41.171087 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c10a3d5-f659-4cee-a979-8ce2f2cb1738 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:41.172454 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c10a3d5-f659-4cee-a979-8ce2f2cb1738 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:41.198581 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 405/828] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:06:40 2022] POST /identity/v3/endpoints => generated 346 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:06:41.200763 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a82f2-4d8f-41ea-8352-c29c9ae64b85 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:41.201176 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a82f2-4d8f-41ea-8352-c29c9ae64b85 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:41.201822 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a82f2-4d8f-41ea-8352-c29c9ae64b85 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:41.204624 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 406/829] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:41.211283 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce257166-487c-423d-9389-d5b457b265a9 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:41.211784 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce257166-487c-423d-9389-d5b457b265a9 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:41.212621 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce257166-487c-423d-9389-d5b457b265a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:41.236792 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-ce257166-487c-423d-9389-d5b457b265a9 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:41.238746 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-ce257166-487c-423d-9389-d5b457b265a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:41.242204 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee0a7cb-4b2c-4c1c-8913-58f1bf2d8f9f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:41.288034 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec020c5-6966-41cc-a36c-ff4cb21408d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:41.288698 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec020c5-6966-41cc-a36c-ff4cb21408d0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:41.288973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec020c5-6966-41cc-a36c-ff4cb21408d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:41.289270 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec020c5-6966-41cc-a36c-ff4cb21408d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:41.289968 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec020c5-6966-41cc-a36c-ff4cb21408d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:41.290312 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec020c5-6966-41cc-a36c-ff4cb21408d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:41.291706 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec020c5-6966-41cc-a36c-ff4cb21408d0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:41.306146 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 424/830] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:41.319132 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec020c5-6966-41cc-a36c-ff4cb21408d0 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:41.323716 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 407/831] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 05:06:41.392075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:41.428217 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec020c5-6966-41cc-a36c-ff4cb21408d0 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:41.463309 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b1bbda-d962-45d0-b3cc-e2258020e2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:41.463856 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1bbda-d962-45d0-b3cc-e2258020e2b0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:41.464126 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1bbda-d962-45d0-b3cc-e2258020e2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:41.464388 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1bbda-d962-45d0-b3cc-e2258020e2b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:41.471234 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b1bbda-d962-45d0-b3cc-e2258020e2b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:41.472669 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b1bbda-d962-45d0-b3cc-e2258020e2b0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:41.483918 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-25b1bbda-d962-45d0-b3cc-e2258020e2b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:41.483918 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:41.483918 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:41.483918 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:41.483918 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:41.483918 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:41.483918 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:41.483918 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:41.483918 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:41.483918 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:41.483918 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:41.483918 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:41.483918 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 05:06:41.483918 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 05:06:41.483918 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 05:06:41.484832 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 05:06:41.484832 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:41.484832 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:41.484832 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:41.484832 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:41.484832 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:41.484832 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:41.484832 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:41.484832 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:41.484832 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:41.484832 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:41.484832 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:41.484832 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:41.484832 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:41.484832 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:41.484832 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:41.484832 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:41.485751 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 05:06:41.485751 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 05:06:41.485751 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 05:06:41.485751 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 05:06:41.485751 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 05:06:41.485751 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 05:06:41.485751 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:41.485751 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:41.485751 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 425/832] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:06:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:41.491024 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce257166-487c-423d-9389-d5b457b265a9 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:41.574878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:41.592476 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b1bbda-d962-45d0-b3cc-e2258020e2b0 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:41.595648 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce257166-487c-423d-9389-d5b457b265a9 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:41.631551 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf79b00a-cb4d-4c5f-9c57-f740597fc020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:41.632308 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf79b00a-cb4d-4c5f-9c57-f740597fc020 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:41.632646 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf79b00a-cb4d-4c5f-9c57-f740597fc020 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:41.632976 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf79b00a-cb4d-4c5f-9c57-f740597fc020 None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:41.633762 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf79b00a-cb4d-4c5f-9c57-f740597fc020 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:41.634160 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf79b00a-cb4d-4c5f-9c57-f740597fc020 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:41.635667 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf79b00a-cb4d-4c5f-9c57-f740597fc020 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:41.653040 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 408/833] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 05:06:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:41.654371 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf79b00a-cb4d-4c5f-9c57-f740597fc020 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:41.708393 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b1bbda-d962-45d0-b3cc-e2258020e2b0 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:41.743909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7dd60a-9d00-4ecb-b4d5-6ffe816fbef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:41.744686 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7dd60a-9d00-4ecb-b4d5-6ffe816fbef5 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:41.745026 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7dd60a-9d00-4ecb-b4d5-6ffe816fbef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:41.745447 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7dd60a-9d00-4ecb-b4d5-6ffe816fbef5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:41.750213 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:41.758320 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf79b00a-cb4d-4c5f-9c57-f740597fc020 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:41.767274 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7dd60a-9d00-4ecb-b4d5-6ffe816fbef5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:06:41.773133 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-7c7dd60a-9d00-4ecb-b4d5-6ffe816fbef5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 05:06:41.773133 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:41.773133 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:41.773133 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:41.773133 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:41.773133 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:41.773133 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:41.773133 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:41.773133 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:41.773133 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:41.773133 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:41.773133 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:41.773133 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:06:41.773133 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:06:41.773133 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:06:41.773133 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:06:41.774270 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:06:41.774270 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:06:41.774270 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:06:41.774270 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:06:41.774270 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:41.774270 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:41.774270 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:41.774270 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:41.774270 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:41.774270 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:41.774270 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:41.774270 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:41.774270 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:41.774270 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:41.774270 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:41.774270 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:41.775329 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:41.775329 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:41.775329 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:41.775329 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:41.775329 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:06:41.775329 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:06:41.775329 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:06:41.775329 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:06:41.775329 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:06:41.775329 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:06:41.775329 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 05:06:41.775329 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:41.778154 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 426/834] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 05:06:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:06:41.789098 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7dd60a-9d00-4ecb-b4d5-6ffe816fbef5 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:41.795626 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6153fc8-01d0-43a3-a549-d741f8f93670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:41.796456 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6153fc8-01d0-43a3-a549-d741f8f93670 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:41.796631 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6153fc8-01d0-43a3-a549-d741f8f93670 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:41.796962 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6153fc8-01d0-43a3-a549-d741f8f93670 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:41.797750 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6153fc8-01d0-43a3-a549-d741f8f93670 None admin] RBAC: Adding query filter params (role.id=c3d5589dd2864be9a0f26f5802d874cd, scope.project.id=e01ce1b1282e44a39d659b87f120995e, user.id=f27665b1574a4db7b7a9771f7d775826) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:41.798141 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6153fc8-01d0-43a3-a549-d741f8f93670 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:41.799696 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6153fc8-01d0-43a3-a549-d741f8f93670 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:41.805626 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 409/835] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:06:41 2022] GET /identity/v3/role_assignments?user.id=f27665b1574a4db7b7a9771f7d775826&scope.project.id=e01ce1b1282e44a39d659b87f120995e&role.id=c3d5589dd2864be9a0f26f5802d874cd => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:06:41.905876 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7dd60a-9d00-4ecb-b4d5-6ffe816fbef5 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:41.945862 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61433b59-83ec-4a1d-9e97-5ce664ef5b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:41.946785 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61433b59-83ec-4a1d-9e97-5ce664ef5b3e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:41.947204 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61433b59-83ec-4a1d-9e97-5ce664ef5b3e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:41.947611 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61433b59-83ec-4a1d-9e97-5ce664ef5b3e None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:41.948424 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61433b59-83ec-4a1d-9e97-5ce664ef5b3e None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:41.948862 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61433b59-83ec-4a1d-9e97-5ce664ef5b3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:41.950481 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61433b59-83ec-4a1d-9e97-5ce664ef5b3e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:41.959874 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 427/836] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Aug 31 05:06:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:41.965157 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6153fc8-01d0-43a3-a549-d741f8f93670 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:41.976703 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:42.074391 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6153fc8-01d0-43a3-a549-d741f8f93670 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:42.111479 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c5fb2c-e72b-4fab-bdf3-394ed23ba26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:42.112276 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5fb2c-e72b-4fab-bdf3-394ed23ba26e None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:42.112644 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5fb2c-e72b-4fab-bdf3-394ed23ba26e None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:42.113005 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5fb2c-e72b-4fab-bdf3-394ed23ba26e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:42.124532 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c5fb2c-e72b-4fab-bdf3-394ed23ba26e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:42.126122 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c5fb2c-e72b-4fab-bdf3-394ed23ba26e None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:42.140501 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-69c5fb2c-e72b-4fab-bdf3-394ed23ba26e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:42.140501 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:42.140501 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:42.140501 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:42.140501 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:42.140501 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:42.140501 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:42.140501 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:42.140501 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:42.140501 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:42.140501 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:42.140501 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:42.140501 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:42.140501 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:42.140501 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:42.140501 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:42.141620 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:42.141620 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:42.141620 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:42.141620 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:42.141620 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:42.141620 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:42.141620 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:42.141620 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:42.141620 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:42.141620 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:42.141620 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:42.141620 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:42.141620 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:42.141620 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:42.141620 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:42.141620 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:42.141620 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:42.142767 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:42.142767 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:42.142767 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:42.144082 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 410/837] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:42.151388 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61433b59-83ec-4a1d-9e97-5ce664ef5b3e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:42.202153 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:42.254374 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61433b59-83ec-4a1d-9e97-5ce664ef5b3e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:42.291566 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed121864-8332-43e3-9a96-2c97e6d2185a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:42.292177 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed121864-8332-43e3-9a96-2c97e6d2185a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:42.292448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed121864-8332-43e3-9a96-2c97e6d2185a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:42.292708 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed121864-8332-43e3-9a96-2c97e6d2185a None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:42.293339 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed121864-8332-43e3-9a96-2c97e6d2185a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:42.293632 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed121864-8332-43e3-9a96-2c97e6d2185a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:42.295084 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed121864-8332-43e3-9a96-2c97e6d2185a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:42.307751 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 428/838] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:42.315169 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c5fb2c-e72b-4fab-bdf3-394ed23ba26e None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:42.375785 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:42.416854 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c5fb2c-e72b-4fab-bdf3-394ed23ba26e None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:42.449714 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d50048-e5f4-44eb-8838-9584068aad18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:42.450398 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d50048-e5f4-44eb-8838-9584068aad18 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:42.450671 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d50048-e5f4-44eb-8838-9584068aad18 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:42.450981 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d50048-e5f4-44eb-8838-9584068aad18 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:42.460040 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d50048-e5f4-44eb-8838-9584068aad18 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:42.461598 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d50048-e5f4-44eb-8838-9584068aad18 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:42.474228 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-d4d50048-e5f4-44eb-8838-9584068aad18 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 05:06:42.474228 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:42.474228 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:42.474228 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:42.474228 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:42.474228 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:42.474228 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:42.474228 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:42.474228 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:42.474228 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:42.474228 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:42.474228 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:42.474228 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 05:06:42.474228 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 05:06:42.474228 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 05:06:42.474228 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 05:06:42.475265 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:42.475265 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:42.475265 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 05:06:42.475265 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:42.475265 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 05:06:42.475265 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:42.475265 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:42.475265 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:42.475265 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:42.475265 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:42.475265 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:42.475265 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:42.475265 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:42.475265 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:42.475265 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:42.475265 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:42.475265 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:42.475265 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:42.476295 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:42.476295 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:42.476295 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 05:06:42.476295 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 05:06:42.476295 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 05:06:42.476295 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 05:06:42.476295 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 05:06:42.476295 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 05:06:42.476295 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 05:06:42.476295 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:42.476295 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 411/839] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Aug 31 05:06:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:42.481401 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed121864-8332-43e3-9a96-2c97e6d2185a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:42.585850 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed121864-8332-43e3-9a96-2c97e6d2185a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:42.595604 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:42.620613 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec996eb1-b643-4d72-aec1-edeee7bd6b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:42.621220 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec996eb1-b643-4d72-aec1-edeee7bd6b69 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:42.621493 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec996eb1-b643-4d72-aec1-edeee7bd6b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:42.621758 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec996eb1-b643-4d72-aec1-edeee7bd6b69 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:42.622682 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec996eb1-b643-4d72-aec1-edeee7bd6b69 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:42.622981 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec996eb1-b643-4d72-aec1-edeee7bd6b69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:42.624398 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec996eb1-b643-4d72-aec1-edeee7bd6b69 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:42.647920 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 429/840] 10.222.0.36 () {56 vars in 1149 bytes} [Wed Aug 31 05:06:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:42.653578 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d50048-e5f4-44eb-8838-9584068aad18 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:42.757147 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d50048-e5f4-44eb-8838-9584068aad18 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:42.774638 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:42.791723 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22125480-0cf2-4656-b8de-d537e16e54bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:42.792323 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22125480-0cf2-4656-b8de-d537e16e54bc None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:42.792593 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22125480-0cf2-4656-b8de-d537e16e54bc None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:42.792859 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22125480-0cf2-4656-b8de-d537e16e54bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:42.861394 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22125480-0cf2-4656-b8de-d537e16e54bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:42.862900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22125480-0cf2-4656-b8de-d537e16e54bc None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:42.953376 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9571dc1-0435-4327-9e68-f34e20a13e7b None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:42.953998 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9571dc1-0435-4327-9e68-f34e20a13e7b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:42.954574 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9571dc1-0435-4327-9e68-f34e20a13e7b None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:42.956479 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 430/841] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 05:06:42.962392 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917b647-0873-401a-b296-d58d3f046dd8 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:42.963032 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917b647-0873-401a-b296-d58d3f046dd8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:42.963503 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917b647-0873-401a-b296-d58d3f046dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:42.964630 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 412/842] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:06:42 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 05:06:42.972329 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22125480-0cf2-4656-b8de-d537e16e54bc None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:42.983300 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-c917b647-0873-401a-b296-d58d3f046dd8 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:42.984883 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c917b647-0873-401a-b296-d58d3f046dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:42.992231 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:43.056583 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 431/843] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:42 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 05:06:43.083117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22125480-0cf2-4656-b8de-d537e16e54bc None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:43.116284 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f32f4-b37c-43f1-a581-ea34845b594c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:43.116900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f32f4-b37c-43f1-a581-ea34845b594c None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:43.117172 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f32f4-b37c-43f1-a581-ea34845b594c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:43.117434 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f32f4-b37c-43f1-a581-ea34845b594c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:43.127858 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538f32f4-b37c-43f1-a581-ea34845b594c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:43.129306 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538f32f4-b37c-43f1-a581-ea34845b594c None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:43.143727 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-538f32f4-b37c-43f1-a581-ea34845b594c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:43.143727 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:43.143727 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:43.143727 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:43.143727 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:43.143727 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:43.143727 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:43.143727 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:43.143727 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:43.143727 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:43.143727 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:43.143727 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:43.143727 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:43.143727 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:43.143727 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:43.143727 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:43.144660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:43.144660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:43.144660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:43.144660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:43.144660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:43.144660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:43.144660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:43.144660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:43.144660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:43.144660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:43.144660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:43.144660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:43.144660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:43.144660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:43.144660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:43.144660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:43.144660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:43.145554 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:43.145554 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:43.145554 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:43.145554 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 413/844] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:43.150452 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917b647-0873-401a-b296-d58d3f046dd8 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:43.246885 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:43.252605 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917b647-0873-401a-b296-d58d3f046dd8 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:43.286511 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa57c32-d5fc-40c0-a8b4-f48ee26e22ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:43.287126 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa57c32-d5fc-40c0-a8b4-f48ee26e22ce None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:43.287398 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa57c32-d5fc-40c0-a8b4-f48ee26e22ce None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:43.287663 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa57c32-d5fc-40c0-a8b4-f48ee26e22ce None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:43.288301 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa57c32-d5fc-40c0-a8b4-f48ee26e22ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:43.288593 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa57c32-d5fc-40c0-a8b4-f48ee26e22ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:43.289958 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa57c32-d5fc-40c0-a8b4-f48ee26e22ce None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:43.301880 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 432/845] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:06:43.307419 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f32f4-b37c-43f1-a581-ea34845b594c None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:43.331346 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa57c32-d5fc-40c0-a8b4-f48ee26e22ce None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:43.381809 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:43.415142 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f32f4-b37c-43f1-a581-ea34845b594c None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:43.436926 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa57c32-d5fc-40c0-a8b4-f48ee26e22ce None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:43.451800 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2443e-4ea5-4c75-a382-d5c1bb9a84fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:43.452620 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2443e-4ea5-4c75-a382-d5c1bb9a84fb None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:43.453036 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2443e-4ea5-4c75-a382-d5c1bb9a84fb None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:43.453441 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2443e-4ea5-4c75-a382-d5c1bb9a84fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:43.460367 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c2443e-4ea5-4c75-a382-d5c1bb9a84fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:43.461816 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c2443e-4ea5-4c75-a382-d5c1bb9a84fb None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:43.470747 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c70bc8-f96d-4826-a8c1-220c433d0226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:43.471371 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c70bc8-f96d-4826-a8c1-220c433d0226 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:43.471642 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c70bc8-f96d-4826-a8c1-220c433d0226 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:43.471906 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c70bc8-f96d-4826-a8c1-220c433d0226 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:43.473451 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-27c2443e-4ea5-4c75-a382-d5c1bb9a84fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:43.473451 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:43.473451 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:43.473451 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:43.473451 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:43.473451 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:43.473451 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:43.473451 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:43.473451 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:43.473451 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:43.473451 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:43.473451 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:43.473451 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 05:06:43.473451 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 05:06:43.473451 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 05:06:43.474335 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 05:06:43.474335 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:43.474335 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:43.474335 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:43.474335 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:43.474335 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:43.474335 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:43.474335 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:43.474335 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:43.474335 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:43.474335 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:43.474335 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:43.474335 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:43.474335 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:43.474335 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:43.474335 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:43.474335 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:43.475258 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 05:06:43.475258 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 05:06:43.475258 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 05:06:43.475258 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 05:06:43.475258 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 05:06:43.475258 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 05:06:43.475258 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:43.475258 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:43.475258 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 414/846] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:06:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:43.476305 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007969b-b4a7-44fd-9376-55da1066309a None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:43.476305 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007969b-b4a7-44fd-9376-55da1066309a None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:43.476305 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007969b-b4a7-44fd-9376-55da1066309a None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:43.481931 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 415/847] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:43.482772 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a007969b-b4a7-44fd-9376-55da1066309a None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:43.488975 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c70bc8-f96d-4826-a8c1-220c433d0226 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:06:43.494533 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-e8c70bc8-f96d-4826-a8c1-220c433d0226 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:06:43.494533 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:43.494533 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:43.494533 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:43.494533 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:43.494533 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:43.494533 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:43.494533 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:43.494533 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:43.494533 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:43.494533 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:43.494533 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:43.494533 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:06:43.494533 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:06:43.494533 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:06:43.494533 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:06:43.495469 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:06:43.495469 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:06:43.495469 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:06:43.495469 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:06:43.495469 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:43.495469 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:43.495469 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:43.495469 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:43.495469 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:43.495469 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:43.495469 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:43.495469 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:43.495469 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:43.495469 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:43.495469 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:43.495469 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:43.496317 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:43.496317 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:43.496317 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:43.496317 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:43.496317 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:06:43.496317 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:06:43.496317 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:06:43.496317 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:06:43.496317 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:06:43.496317 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:06:43.496317 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:06:43.496317 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:43.496317 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 433/848] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 05:06:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:06:43.497153 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c688d4c-2fd5-4862-a68e-06c0ff37d5b1 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:43.497153 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c688d4c-2fd5-4862-a68e-06c0ff37d5b1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:43.498212 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c688d4c-2fd5-4862-a68e-06c0ff37d5b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:43.516113 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-8c688d4c-2fd5-4862-a68e-06c0ff37d5b1 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:43.517392 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8c688d4c-2fd5-4862-a68e-06c0ff37d5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:43.529911 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:43.587642 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 434/849] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:43 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:06:43.588592 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c688d4c-2fd5-4862-a68e-06c0ff37d5b1 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:43.603513 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a007969b-b4a7-44fd-9376-55da1066309a None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:43.633435 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:43.640110 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da02f0a-af34-4f5d-8de1-916d246ee05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:43.641095 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da02f0a-af34-4f5d-8de1-916d246ee05b None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:43.641613 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da02f0a-af34-4f5d-8de1-916d246ee05b None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:43.642125 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da02f0a-af34-4f5d-8de1-916d246ee05b None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:43.643728 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da02f0a-af34-4f5d-8de1-916d246ee05b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:43.644321 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da02f0a-af34-4f5d-8de1-916d246ee05b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:43.646330 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da02f0a-af34-4f5d-8de1-916d246ee05b None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:43.666049 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 416/850] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 05:06:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:43.672552 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da02f0a-af34-4f5d-8de1-916d246ee05b None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:43.722921 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c688d4c-2fd5-4862-a68e-06c0ff37d5b1 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:43.772049 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313644a9-81c8-4325-ba49-cc5c72b8f684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:43.773113 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313644a9-81c8-4325-ba49-cc5c72b8f684 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:43.773630 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313644a9-81c8-4325-ba49-cc5c72b8f684 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:43.774148 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313644a9-81c8-4325-ba49-cc5c72b8f684 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:43.775271 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313644a9-81c8-4325-ba49-cc5c72b8f684 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:43.776132 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313644a9-81c8-4325-ba49-cc5c72b8f684 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:43.778156 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313644a9-81c8-4325-ba49-cc5c72b8f684 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:43.785121 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da02f0a-af34-4f5d-8de1-916d246ee05b None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:43.789932 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 435/851] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:06:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:43.791229 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752334e1-34d7-4c13-8938-e854db784d56 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:43.791511 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752334e1-34d7-4c13-8938-e854db784d56 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:43.791779 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752334e1-34d7-4c13-8938-e854db784d56 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:43.793582 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 436/852] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:43.796144 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752334e1-34d7-4c13-8938-e854db784d56 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:43.807382 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:43.823134 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ebd6e7-58b6-4170-94e4-ea75f638d055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:43.823735 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ebd6e7-58b6-4170-94e4-ea75f638d055 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:43.824003 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ebd6e7-58b6-4170-94e4-ea75f638d055 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:43.824257 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ebd6e7-58b6-4170-94e4-ea75f638d055 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:43.824976 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ebd6e7-58b6-4170-94e4-ea75f638d055 None admin] RBAC: Adding query filter params (role.id=207bf98e31774cc0b21249de6d96155b, scope.project.id=e01ce1b1282e44a39d659b87f120995e, user.id=09bf6db88ce8494a937211a814c9ff84) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:43.825369 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ebd6e7-58b6-4170-94e4-ea75f638d055 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:43.826823 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ebd6e7-58b6-4170-94e4-ea75f638d055 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:43.832792 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 417/853] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:06:43 2022] GET /identity/v3/role_assignments?user.id=09bf6db88ce8494a937211a814c9ff84&scope.project.id=e01ce1b1282e44a39d659b87f120995e&role.id=207bf98e31774cc0b21249de6d96155b => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:06:43.834021 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e1708-4758-4032-8ae9-a8deefdef774 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:43.834294 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e1708-4758-4032-8ae9-a8deefdef774 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:43.834558 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e1708-4758-4032-8ae9-a8deefdef774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:43.855563 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-7b2e1708-4758-4032-8ae9-a8deefdef774 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:43.857007 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-7b2e1708-4758-4032-8ae9-a8deefdef774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:43.914714 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752334e1-34d7-4c13-8938-e854db784d56 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:43.921528 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:43.942285 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 418/854] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:43 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:06:43.943066 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2e1708-4758-4032-8ae9-a8deefdef774 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:43.958031 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d182ac79-0db9-4863-80ef-7a048008a1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:43.958031 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182ac79-0db9-4863-80ef-7a048008a1c0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:43.958031 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182ac79-0db9-4863-80ef-7a048008a1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:43.958031 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182ac79-0db9-4863-80ef-7a048008a1c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:43.971148 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d182ac79-0db9-4863-80ef-7a048008a1c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:43.972791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d182ac79-0db9-4863-80ef-7a048008a1c0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:43.988935 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-d182ac79-0db9-4863-80ef-7a048008a1c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:43.988935 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:43.988935 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:43.988935 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:43.988935 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:43.988935 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:43.988935 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:43.988935 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:43.988935 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:43.988935 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:43.988935 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:43.988935 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:43.988935 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:43.988935 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:43.988935 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:43.988935 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:43.990157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:43.990157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:43.990157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:43.990157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:43.990157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:43.990157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:43.990157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:43.990157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:43.990157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:43.990157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:43.990157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:43.990157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:43.990157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:43.990157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:43.990157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:43.990157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:43.990157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:43.991293 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:43.991293 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:43.991293 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:43.991293 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 437/855] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:43.996778 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d182ac79-0db9-4863-80ef-7a048008a1c0 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:44.061720 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:44.067105 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2e1708-4758-4032-8ae9-a8deefdef774 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:44.107376 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e959e-831a-4d79-bb01-ac74a5c23b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:44.108362 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e959e-831a-4d79-bb01-ac74a5c23b4c None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:44.108814 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e959e-831a-4d79-bb01-ac74a5c23b4c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:44.109087 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e959e-831a-4d79-bb01-ac74a5c23b4c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:44.109672 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60e959e-831a-4d79-bb01-ac74a5c23b4c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:44.111085 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60e959e-831a-4d79-bb01-ac74a5c23b4c None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:44.111198 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d182ac79-0db9-4863-80ef-7a048008a1c0 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:44.120900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-b60e959e-831a-4d79-bb01-ac74a5c23b4c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 05:06:44.120900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:44.120900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 05:06:44.120900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 05:06:44.120900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 05:06:44.120900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 05:06:44.120900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 05:06:44.120900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:06:44.120900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 05:06:44.120900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:44.120900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 05:06:44.120900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:44.120900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:44.120900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:44.120900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:44.120900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:44.120900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:44.122210 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:44.122210 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:44.122210 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:44.122210 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:44.122210 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:44.122210 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:44.122210 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 05:06:44.122210 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 05:06:44.122210 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 05:06:44.122210 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 05:06:44.122210 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:44.122210 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:44.122210 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:44.122210 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:44.122210 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:44.122210 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:44.123289 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:44.123289 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:44.123289 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:44.123289 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:44.123289 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:44.123289 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:44.123289 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:44.123289 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:44.123289 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:44.123289 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:44.123289 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 05:06:44.123289 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:06:44.123289 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 05:06:44.123289 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:44.123289 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 419/856] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 05:06:43 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:44.130710 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e959e-831a-4d79-bb01-ac74a5c23b4c None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:44.155339 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632baf1c-a8bb-469f-a071-e3a475d155b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:44.155949 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632baf1c-a8bb-469f-a071-e3a475d155b7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:44.156219 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632baf1c-a8bb-469f-a071-e3a475d155b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:44.156482 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632baf1c-a8bb-469f-a071-e3a475d155b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:44.157483 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632baf1c-a8bb-469f-a071-e3a475d155b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:44.157483 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632baf1c-a8bb-469f-a071-e3a475d155b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:44.162026 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632baf1c-a8bb-469f-a071-e3a475d155b7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:44.176365 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 438/857] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:44.182187 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632baf1c-a8bb-469f-a071-e3a475d155b7 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:44.214667 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:44.249269 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e959e-831a-4d79-bb01-ac74a5c23b4c None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:44.286034 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e8606-c6bf-4f0d-92ca-ce723199913e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:44.286743 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e8606-c6bf-4f0d-92ca-ce723199913e None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:44.287012 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e8606-c6bf-4f0d-92ca-ce723199913e None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:44.287338 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e8606-c6bf-4f0d-92ca-ce723199913e None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:44.287959 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9e8606-c6bf-4f0d-92ca-ce723199913e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:44.288253 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9e8606-c6bf-4f0d-92ca-ce723199913e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:44.289509 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632baf1c-a8bb-469f-a071-e3a475d155b7 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:44.289612 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9e8606-c6bf-4f0d-92ca-ce723199913e None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:44.299145 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 420/858] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 05:06:44 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:44.299145 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e8606-c6bf-4f0d-92ca-ce723199913e None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:44.312387 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:44.327807 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c761f-c310-46f6-ade8-f38fda848e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:44.328523 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c761f-c310-46f6-ade8-f38fda848e42 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:44.328863 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c761f-c310-46f6-ade8-f38fda848e42 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:44.329195 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c761f-c310-46f6-ade8-f38fda848e42 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:44.336473 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253c761f-c310-46f6-ade8-f38fda848e42 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:44.337987 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253c761f-c310-46f6-ade8-f38fda848e42 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:44.350021 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-253c761f-c310-46f6-ade8-f38fda848e42 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 05:06:44.350021 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:44.350021 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:44.350021 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:44.350021 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:44.350021 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:44.350021 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:44.350021 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:44.350021 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:44.350021 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:44.350021 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:44.350021 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:44.350021 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 05:06:44.350021 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 05:06:44.350021 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 05:06:44.350021 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 05:06:44.351159 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:44.351159 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:44.351159 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 05:06:44.351159 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:44.351159 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 05:06:44.351159 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:44.351159 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:44.351159 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:44.351159 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:44.351159 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:44.351159 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:44.351159 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:44.351159 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:44.351159 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:44.351159 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:44.351159 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:44.351159 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:44.351159 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:44.352352 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:44.352352 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:44.352352 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 05:06:44.352352 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 05:06:44.352352 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 05:06:44.352352 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 05:06:44.352352 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 05:06:44.352352 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 05:06:44.352352 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 05:06:44.352352 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:44.353415 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 439/859] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 05:06:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:44.354100 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c761f-c310-46f6-ade8-f38fda848e42 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:44.409418 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e8606-c6bf-4f0d-92ca-ce723199913e None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:44.423873 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:44.445980 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11827461-c436-44e3-bfa1-d4c8ea3a251c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:44.446627 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11827461-c436-44e3-bfa1-d4c8ea3a251c None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:44.446903 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11827461-c436-44e3-bfa1-d4c8ea3a251c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:44.447167 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11827461-c436-44e3-bfa1-d4c8ea3a251c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:44.447749 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11827461-c436-44e3-bfa1-d4c8ea3a251c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:44.449118 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11827461-c436-44e3-bfa1-d4c8ea3a251c None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:44.457994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-11827461-c436-44e3-bfa1-d4c8ea3a251c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 05:06:44.457994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:44.457994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 05:06:44.457994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 05:06:44.457994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 05:06:44.457994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 05:06:44.457994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 05:06:44.457994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:06:44.457994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 05:06:44.457994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:44.457994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 05:06:44.457994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:44.457994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:44.457994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:44.457994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:44.457994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:44.459015 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:44.459015 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:44.459015 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:44.459015 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:44.459015 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:44.459015 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:44.459015 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:44.459015 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 05:06:44.459015 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 05:06:44.459015 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 05:06:44.459015 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 05:06:44.459015 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:44.459015 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:44.459015 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:44.459015 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:44.459015 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:44.459015 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:44.460566 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:44.460566 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:44.460566 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:44.460566 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:44.460566 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:44.460566 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:44.460566 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:44.460566 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:44.460566 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:44.460566 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:44.460566 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 05:06:44.460566 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:06:44.460566 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 05:06:44.460566 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:44.460566 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 421/860] 10.222.0.36 () {56 vars in 1132 bytes} [Wed Aug 31 05:06:44 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:44.461381 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11827461-c436-44e3-bfa1-d4c8ea3a251c None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:44.467831 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c761f-c310-46f6-ade8-f38fda848e42 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:44.503922 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e505d-3f56-46b0-9566-240cfbe2dff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:44.504881 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e505d-3f56-46b0-9566-240cfbe2dff4 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:44.505351 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e505d-3f56-46b0-9566-240cfbe2dff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:44.505843 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e505d-3f56-46b0-9566-240cfbe2dff4 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:44.506561 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e505d-3f56-46b0-9566-240cfbe2dff4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:44.506925 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e505d-3f56-46b0-9566-240cfbe2dff4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:44.508352 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e505d-3f56-46b0-9566-240cfbe2dff4 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:44.513739 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 440/861] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 05:06:44 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:44.514612 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e505d-3f56-46b0-9566-240cfbe2dff4 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:44.581503 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:44.581688 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11827461-c436-44e3-bfa1-d4c8ea3a251c None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:44.620786 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce672aff-a736-48bd-93f4-ed27356ff48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:44.621843 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce672aff-a736-48bd-93f4-ed27356ff48a None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:44.622571 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce672aff-a736-48bd-93f4-ed27356ff48a None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:44.623092 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce672aff-a736-48bd-93f4-ed27356ff48a None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:44.624069 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce672aff-a736-48bd-93f4-ed27356ff48a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:44.624589 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce672aff-a736-48bd-93f4-ed27356ff48a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:44.626389 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce672aff-a736-48bd-93f4-ed27356ff48a None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:44.649113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 422/862] 10.222.0.36 () {56 vars in 1135 bytes} [Wed Aug 31 05:06:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:44.654822 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce672aff-a736-48bd-93f4-ed27356ff48a None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:44.671087 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e505d-3f56-46b0-9566-240cfbe2dff4 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:44.699200 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:44.710401 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b43d085-02bb-4ba9-a917-c1483fcf9ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:44.711444 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b43d085-02bb-4ba9-a917-c1483fcf9ff3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:44.711890 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b43d085-02bb-4ba9-a917-c1483fcf9ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:44.712321 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b43d085-02bb-4ba9-a917-c1483fcf9ff3 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:44.713110 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b43d085-02bb-4ba9-a917-c1483fcf9ff3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:44.713573 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b43d085-02bb-4ba9-a917-c1483fcf9ff3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:44.715413 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b43d085-02bb-4ba9-a917-c1483fcf9ff3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:44.721072 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 441/863] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Aug 31 05:06:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:44.726982 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b43d085-02bb-4ba9-a917-c1483fcf9ff3 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:44.776217 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce672aff-a736-48bd-93f4-ed27356ff48a None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:44.825741 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2c6d0-cbbc-4de3-bfbf-5cae0367140b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:44.826773 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2c6d0-cbbc-4de3-bfbf-5cae0367140b None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:44.827683 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2c6d0-cbbc-4de3-bfbf-5cae0367140b None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:44.828148 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2c6d0-cbbc-4de3-bfbf-5cae0367140b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:44.854739 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b43d085-02bb-4ba9-a917-c1483fcf9ff3 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:44.865744 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:44.894552 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c4822-160e-4b9d-a3cd-0c34aadda94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:44.895304 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c4822-160e-4b9d-a3cd-0c34aadda94e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:44.895611 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c4822-160e-4b9d-a3cd-0c34aadda94e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:44.895917 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c4822-160e-4b9d-a3cd-0c34aadda94e None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:44.896560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536c4822-160e-4b9d-a3cd-0c34aadda94e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:44.897057 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536c4822-160e-4b9d-a3cd-0c34aadda94e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:44.898303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536c4822-160e-4b9d-a3cd-0c34aadda94e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:44.903539 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 442/864] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Aug 31 05:06:44 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:44.925731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa2c6d0-cbbc-4de3-bfbf-5cae0367140b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:44.927719 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa2c6d0-cbbc-4de3-bfbf-5cae0367140b None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:44.967400 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:45.045307 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 423/865] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:06:44 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 05:06:45.050895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c4822-160e-4b9d-a3cd-0c34aadda94e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:45.162646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c4822-160e-4b9d-a3cd-0c34aadda94e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:45.200744 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694cbc59-ff31-4989-affb-7e8f06cb1001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:45.201601 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694cbc59-ff31-4989-affb-7e8f06cb1001 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:45.201978 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694cbc59-ff31-4989-affb-7e8f06cb1001 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:45.202345 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694cbc59-ff31-4989-affb-7e8f06cb1001 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:45.213606 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694cbc59-ff31-4989-affb-7e8f06cb1001 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:45.215077 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694cbc59-ff31-4989-affb-7e8f06cb1001 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:45.229288 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-694cbc59-ff31-4989-affb-7e8f06cb1001 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:45.229288 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:45.229288 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:45.229288 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:45.229288 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:45.229288 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:45.229288 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:45.229288 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:45.229288 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:45.229288 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:45.229288 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:45.229288 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:45.229288 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:45.229288 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:45.229288 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:45.229288 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:45.230256 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:45.230256 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:45.230256 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:45.230256 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:45.230256 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:45.230256 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:45.230256 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:45.230256 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:45.230256 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:45.230256 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:45.230256 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:45.230256 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:45.230256 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:45.230256 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:45.230256 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:45.230256 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:45.230256 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:45.231608 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:45.231608 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:45.231608 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:45.231608 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 443/866] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:45.236518 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2c6d0-cbbc-4de3-bfbf-5cae0367140b None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:45.337503 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2c6d0-cbbc-4de3-bfbf-5cae0367140b None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:45.372414 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d2767c-07a8-48a0-8fd5-8a967ae0a061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:45.373026 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d2767c-07a8-48a0-8fd5-8a967ae0a061 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:45.373304 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d2767c-07a8-48a0-8fd5-8a967ae0a061 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:45.373572 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d2767c-07a8-48a0-8fd5-8a967ae0a061 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:45.374319 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d2767c-07a8-48a0-8fd5-8a967ae0a061 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:45.374621 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d2767c-07a8-48a0-8fd5-8a967ae0a061 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:45.375995 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d2767c-07a8-48a0-8fd5-8a967ae0a061 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:45.388211 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 424/867] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:45.390347 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:45.393967 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694cbc59-ff31-4989-affb-7e8f06cb1001 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:45.421684 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:45.502250 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694cbc59-ff31-4989-affb-7e8f06cb1001 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:45.536705 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58270d26-3dad-466a-94db-0d5c38b131ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:45.537340 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58270d26-3dad-466a-94db-0d5c38b131ae None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:45.537618 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58270d26-3dad-466a-94db-0d5c38b131ae None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:45.537885 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58270d26-3dad-466a-94db-0d5c38b131ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:45.544743 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58270d26-3dad-466a-94db-0d5c38b131ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:45.546278 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58270d26-3dad-466a-94db-0d5c38b131ae None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:45.557708 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-58270d26-3dad-466a-94db-0d5c38b131ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:45.557708 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:45.557708 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:45.557708 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:45.557708 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:45.557708 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:45.557708 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:45.557708 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:45.557708 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:45.557708 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:45.557708 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:45.557708 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:45.557708 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 05:06:45.557708 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 05:06:45.557708 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 05:06:45.559861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 05:06:45.559861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:45.559861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:45.559861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:45.559861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:45.559861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:45.559861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:45.559861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:45.559861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:45.559861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:45.559861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:45.559861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:45.559861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:45.559861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:45.559861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:45.559861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:45.559861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:45.561899 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 05:06:45.561899 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 05:06:45.561899 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 05:06:45.561899 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 05:06:45.561899 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 05:06:45.561899 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 05:06:45.561899 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:45.561899 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:45.561899 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 444/868] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:06:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:45.567346 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d2767c-07a8-48a0-8fd5-8a967ae0a061 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:45.677154 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d2767c-07a8-48a0-8fd5-8a967ae0a061 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:45.711744 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b448d5dc-ce73-4905-90f3-b5f9086dfb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:45.712403 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b448d5dc-ce73-4905-90f3-b5f9086dfb2c None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:45.712677 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b448d5dc-ce73-4905-90f3-b5f9086dfb2c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:45.712945 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b448d5dc-ce73-4905-90f3-b5f9086dfb2c None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:45.713682 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b448d5dc-ce73-4905-90f3-b5f9086dfb2c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:45.713987 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b448d5dc-ce73-4905-90f3-b5f9086dfb2c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:45.715456 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b448d5dc-ce73-4905-90f3-b5f9086dfb2c None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:45.727939 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 425/869] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 05:06:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:45.734125 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58270d26-3dad-466a-94db-0d5c38b131ae None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:45.828437 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:45.846103 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58270d26-3dad-466a-94db-0d5c38b131ae None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:45.880980 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcb3c63-4584-4bf9-b3ea-6082e51bcd42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:45.881586 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb3c63-4584-4bf9-b3ea-6082e51bcd42 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:45.881862 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb3c63-4584-4bf9-b3ea-6082e51bcd42 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:45.882183 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb3c63-4584-4bf9-b3ea-6082e51bcd42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:45.882907 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcb3c63-4584-4bf9-b3ea-6082e51bcd42 None admin] RBAC: Adding query filter params (role.id=c3d5589dd2864be9a0f26f5802d874cd, scope.project.id=e01ce1b1282e44a39d659b87f120995e, user.id=d9b82080a73d4c4a8ec08b0e67d6d11c) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:45.883211 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcb3c63-4584-4bf9-b3ea-6082e51bcd42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:45.884653 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcb3c63-4584-4bf9-b3ea-6082e51bcd42 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:45.890225 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc4479-d110-4e13-a9a4-8576b02bd828 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:45.890580 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc4479-d110-4e13-a9a4-8576b02bd828 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:45.890736 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc4479-d110-4e13-a9a4-8576b02bd828 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:45.892139 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 445/870] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:06:45 2022] GET /identity/v3/role_assignments?user.id=d9b82080a73d4c4a8ec08b0e67d6d11c&scope.project.id=e01ce1b1282e44a39d659b87f120995e&role.id=c3d5589dd2864be9a0f26f5802d874cd => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:45.892379 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 426/871] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:45.899481 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf2e766-a24d-4a2e-b992-7b51d593a219 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:45.899801 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf2e766-a24d-4a2e-b992-7b51d593a219 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:45.900016 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf2e766-a24d-4a2e-b992-7b51d593a219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:45.920418 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-3cf2e766-a24d-4a2e-b992-7b51d593a219 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:45.921632 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3cf2e766-a24d-4a2e-b992-7b51d593a219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:45.997083 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 446/872] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:45 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:06:46.259956 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc4479-d110-4e13-a9a4-8576b02bd828 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:46.315415 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:46.360555 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc4479-d110-4e13-a9a4-8576b02bd828 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:46.394640 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f5ea2f-e057-4b46-a209-3fb5a82d4789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:46.395322 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5ea2f-e057-4b46-a209-3fb5a82d4789 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:46.395643 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5ea2f-e057-4b46-a209-3fb5a82d4789 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:46.395911 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5ea2f-e057-4b46-a209-3fb5a82d4789 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:46.406781 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f5ea2f-e057-4b46-a209-3fb5a82d4789 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:06:46.411643 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-76f5ea2f-e057-4b46-a209-3fb5a82d4789 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 05:06:46.411643 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:46.411643 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:46.411643 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:46.411643 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:46.411643 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:46.411643 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:46.411643 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:46.411643 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:46.411643 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:46.411643 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:46.411643 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:46.411643 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:06:46.411643 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:06:46.411643 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:06:46.411643 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:06:46.412789 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:06:46.412789 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:06:46.412789 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:06:46.412789 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:06:46.412789 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:46.412789 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:46.412789 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:46.412789 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:46.412789 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:46.412789 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:46.412789 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:46.412789 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:46.412789 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:46.412789 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:46.412789 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:46.412789 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:46.413812 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:46.413812 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:46.413812 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:46.413812 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:46.413812 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:06:46.413812 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:06:46.413812 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:06:46.413812 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:06:46.413812 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:06:46.413812 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:06:46.413812 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 05:06:46.413812 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:46.413812 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 427/873] 10.222.0.36 () {56 vars in 1120 bytes} [Wed Aug 31 05:06:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:46.419057 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf2e766-a24d-4a2e-b992-7b51d593a219 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:46.476794 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238681b5-06e6-465d-bd64-fb8da1c1105f None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:46.477126 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238681b5-06e6-465d-bd64-fb8da1c1105f None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:46.477948 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238681b5-06e6-465d-bd64-fb8da1c1105f None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:46.480268 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 428/874] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 05:06:46.488149 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33eceb9-5354-4d11-91a0-52d209ed7986 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:46.488149 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33eceb9-5354-4d11-91a0-52d209ed7986 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:46.488149 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33eceb9-5354-4d11-91a0-52d209ed7986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:46.502365 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:46.506077 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-d33eceb9-5354-4d11-91a0-52d209ed7986 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:46.507522 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-d33eceb9-5354-4d11-91a0-52d209ed7986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:46.525196 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf2e766-a24d-4a2e-b992-7b51d593a219 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:46.560559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c74dff-483d-4bc0-b737-dbcc3222a58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:46.561166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c74dff-483d-4bc0-b737-dbcc3222a58b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:46.561439 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c74dff-483d-4bc0-b737-dbcc3222a58b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:46.561702 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c74dff-483d-4bc0-b737-dbcc3222a58b None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:46.562457 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c74dff-483d-4bc0-b737-dbcc3222a58b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:46.562754 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c74dff-483d-4bc0-b737-dbcc3222a58b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:46.564216 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c74dff-483d-4bc0-b737-dbcc3222a58b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:46.573317 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 447/875] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Aug 31 05:06:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:46.582035 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c74dff-483d-4bc0-b737-dbcc3222a58b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:46.582396 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 429/876] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:46 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 05:06:46.681378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c74dff-483d-4bc0-b737-dbcc3222a58b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:46.719044 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dda301-2c55-43ae-9938-cdae9af09532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:46.719667 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dda301-2c55-43ae-9938-cdae9af09532 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:46.719937 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dda301-2c55-43ae-9938-cdae9af09532 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:46.720198 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dda301-2c55-43ae-9938-cdae9af09532 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:46.729071 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:46.731582 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dda301-2c55-43ae-9938-cdae9af09532 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:46.733147 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dda301-2c55-43ae-9938-cdae9af09532 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:46.747577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-c7dda301-2c55-43ae-9938-cdae9af09532 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:46.747577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:46.747577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:46.747577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:46.747577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:46.747577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:46.747577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:46.747577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:46.747577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:46.747577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:46.747577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:46.747577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:46.747577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:46.747577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:46.747577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:46.747577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:46.749254 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:46.749254 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:46.749254 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:46.749254 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:46.749254 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:46.749254 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:46.749254 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:46.749254 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:46.749254 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:46.749254 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:46.749254 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:46.749254 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:46.749254 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:46.749254 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:46.749254 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:46.749254 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:46.749254 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:46.750423 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:46.750423 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:46.750423 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:46.750423 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 448/877] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:06:46.754698 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33eceb9-5354-4d11-91a0-52d209ed7986 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:46.800600 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:46.848918 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dda301-2c55-43ae-9938-cdae9af09532 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:46.860808 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33eceb9-5354-4d11-91a0-52d209ed7986 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:46.897406 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6501c5a-5459-43bc-ace2-9418cc6a75a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:46.898474 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6501c5a-5459-43bc-ace2-9418cc6a75a0 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:46.898971 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6501c5a-5459-43bc-ace2-9418cc6a75a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:46.899424 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6501c5a-5459-43bc-ace2-9418cc6a75a0 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:46.900331 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6501c5a-5459-43bc-ace2-9418cc6a75a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:46.900820 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6501c5a-5459-43bc-ace2-9418cc6a75a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:46.902517 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6501c5a-5459-43bc-ace2-9418cc6a75a0 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:46.915549 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 430/878] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:46.922399 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6501c5a-5459-43bc-ace2-9418cc6a75a0 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:46.969218 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dda301-2c55-43ae-9938-cdae9af09532 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:47.016932 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8c84ce-480f-4eba-bdde-cefd9d7b12b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:47.017555 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c84ce-480f-4eba-bdde-cefd9d7b12b7 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:47.017832 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c84ce-480f-4eba-bdde-cefd9d7b12b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:47.018130 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c84ce-480f-4eba-bdde-cefd9d7b12b7 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:47.018789 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8c84ce-480f-4eba-bdde-cefd9d7b12b7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:47.020159 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8c84ce-480f-4eba-bdde-cefd9d7b12b7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:47.035589 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6501c5a-5459-43bc-ace2-9418cc6a75a0 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:47.036675 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 449/879] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Aug 31 05:06:46 2022] POST /identity/v3/services => generated 251 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:06:47.037717 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f58a6-548a-4c07-b059-5e1606f06100 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:47.038024 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f58a6-548a-4c07-b059-5e1606f06100 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:47.038300 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f58a6-548a-4c07-b059-5e1606f06100 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:47.042081 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 450/880] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:47.044501 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:47.046195 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c5d87-0fbf-4437-ada0-ae94eb75ca9b None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:47.046634 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c5d87-0fbf-4437-ada0-ae94eb75ca9b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:47.047191 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c5d87-0fbf-4437-ada0-ae94eb75ca9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:47.072152 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-553c5d87-0fbf-4437-ada0-ae94eb75ca9b None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:47.073671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-553c5d87-0fbf-4437-ada0-ae94eb75ca9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:47.101563 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d678f3-7d6a-4e0a-9b0e-aba5294daf1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:47.102354 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d678f3-7d6a-4e0a-9b0e-aba5294daf1a None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:47.102928 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d678f3-7d6a-4e0a-9b0e-aba5294daf1a None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:47.103211 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d678f3-7d6a-4e0a-9b0e-aba5294daf1a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:47.115332 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d678f3-7d6a-4e0a-9b0e-aba5294daf1a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:47.117130 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d678f3-7d6a-4e0a-9b0e-aba5294daf1a None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:47.133538 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:47.134964 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-16d678f3-7d6a-4e0a-9b0e-aba5294daf1a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 05:06:47.134964 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:47.134964 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:47.134964 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:47.134964 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:47.134964 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:47.134964 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:47.134964 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:47.134964 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:47.134964 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:47.134964 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:47.134964 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:47.134964 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 05:06:47.134964 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 05:06:47.134964 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 05:06:47.134964 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 05:06:47.136363 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:47.136363 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:47.136363 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 05:06:47.136363 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:47.136363 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 05:06:47.136363 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:47.136363 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:47.136363 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:47.136363 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:47.136363 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:47.136363 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:47.136363 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:47.136363 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:47.136363 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:47.136363 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:47.136363 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:47.136363 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:47.136363 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:47.137558 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:47.137558 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:47.137558 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 05:06:47.137558 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 05:06:47.137558 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 05:06:47.137558 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 05:06:47.137558 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 05:06:47.137558 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 05:06:47.137558 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 05:06:47.137558 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:47.140105 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 431/881] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Aug 31 05:06:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:06:47.145551 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d678f3-7d6a-4e0a-9b0e-aba5294daf1a None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:47.164363 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 451/882] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:47 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:06:47.258648 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d678f3-7d6a-4e0a-9b0e-aba5294daf1a None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:47.293388 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea2577-a18a-44b8-a5f4-1d0cc892a383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:47.294212 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea2577-a18a-44b8-a5f4-1d0cc892a383 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:47.294372 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea2577-a18a-44b8-a5f4-1d0cc892a383 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:47.294606 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea2577-a18a-44b8-a5f4-1d0cc892a383 None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:47.295333 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ea2577-a18a-44b8-a5f4-1d0cc892a383 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:47.295637 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ea2577-a18a-44b8-a5f4-1d0cc892a383 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:47.297100 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ea2577-a18a-44b8-a5f4-1d0cc892a383 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:47.315279 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 432/883] 10.222.0.36 () {56 vars in 1149 bytes} [Wed Aug 31 05:06:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:47.320587 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553c5d87-0fbf-4437-ada0-ae94eb75ca9b None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:47.421836 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553c5d87-0fbf-4437-ada0-ae94eb75ca9b None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:47.456355 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83997d59-99bf-4e67-90a0-7e6097e9c603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:47.457010 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83997d59-99bf-4e67-90a0-7e6097e9c603 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:47.457285 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83997d59-99bf-4e67-90a0-7e6097e9c603 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:47.457547 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83997d59-99bf-4e67-90a0-7e6097e9c603 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:47.476650 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea2577-a18a-44b8-a5f4-1d0cc892a383 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:47.497529 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:47.512234 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:47.538278 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83997d59-99bf-4e67-90a0-7e6097e9c603 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:47.540895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83997d59-99bf-4e67-90a0-7e6097e9c603 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:47.588466 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea2577-a18a-44b8-a5f4-1d0cc892a383 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:47.628003 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2ef0cc-8868-40f8-a63d-01b6e888d16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:47.628609 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ef0cc-8868-40f8-a63d-01b6e888d16e None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:47.628882 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ef0cc-8868-40f8-a63d-01b6e888d16e None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:47.629153 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ef0cc-8868-40f8-a63d-01b6e888d16e None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:47.629810 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2ef0cc-8868-40f8-a63d-01b6e888d16e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:47.631288 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2ef0cc-8868-40f8-a63d-01b6e888d16e None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:47.666110 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 433/884] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Aug 31 05:06:47 2022] POST /identity/v3/services => generated 269 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:06:47.680601 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 452/885] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:06:47 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 05:06:47.686805 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2ef0cc-8868-40f8-a63d-01b6e888d16e None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:47.775873 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:47.810126 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2ef0cc-8868-40f8-a63d-01b6e888d16e None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:47.850498 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87507f27-4aa9-4cff-9a4b-0b72dc7027b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:47.851502 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87507f27-4aa9-4cff-9a4b-0b72dc7027b3 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:47.851997 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87507f27-4aa9-4cff-9a4b-0b72dc7027b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:47.852464 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87507f27-4aa9-4cff-9a4b-0b72dc7027b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:47.863961 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87507f27-4aa9-4cff-9a4b-0b72dc7027b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:47.865766 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87507f27-4aa9-4cff-9a4b-0b72dc7027b3 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:47.881430 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-87507f27-4aa9-4cff-9a4b-0b72dc7027b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:47.881430 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:47.881430 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:47.881430 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:47.881430 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:47.881430 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:47.881430 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:47.881430 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:47.881430 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:47.881430 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:47.881430 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:47.881430 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:47.881430 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:47.881430 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:47.881430 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:47.881430 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:47.882464 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:47.882464 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:47.882464 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:47.882464 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:47.882464 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:47.882464 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:47.882464 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:47.882464 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:47.882464 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:47.882464 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:47.882464 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:47.882464 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:47.882464 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:47.882464 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:47.882464 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:47.882464 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:47.882464 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:47.883357 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:47.883357 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:47.883357 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:47.885430 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 434/886] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:47.891421 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83997d59-99bf-4e67-90a0-7e6097e9c603 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:47.921364 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39039a90-c421-4d79-b907-fdaeb34f93cf None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:47.921722 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39039a90-c421-4d79-b907-fdaeb34f93cf None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:47.921996 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39039a90-c421-4d79-b907-fdaeb34f93cf None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:47.923736 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 435/887] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:47.929088 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d7cb3b-5821-4731-94ab-1999122bb43f None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:47.929372 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d7cb3b-5821-4731-94ab-1999122bb43f None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:47.930386 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d7cb3b-5821-4731-94ab-1999122bb43f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:47.947788 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-d3d7cb3b-5821-4731-94ab-1999122bb43f None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:47.948885 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-d3d7cb3b-5821-4731-94ab-1999122bb43f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:47.978946 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:47.998713 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83997d59-99bf-4e67-90a0-7e6097e9c603 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:48.026183 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 436/888] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:47 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 05:06:48.033517 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ae45e5-1a34-410e-b11f-29129c091195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:48.034416 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae45e5-1a34-410e-b11f-29129c091195 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:48.034885 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae45e5-1a34-410e-b11f-29129c091195 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:48.035356 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae45e5-1a34-410e-b11f-29129c091195 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:48.036182 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ae45e5-1a34-410e-b11f-29129c091195 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:48.036686 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ae45e5-1a34-410e-b11f-29129c091195 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:48.038307 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ae45e5-1a34-410e-b11f-29129c091195 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:48.050561 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 453/889] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:48.056362 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d7cb3b-5821-4731-94ab-1999122bb43f None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:48.118441 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:48.164037 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d7cb3b-5821-4731-94ab-1999122bb43f None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:48.203275 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76b07ec-b021-47e6-a815-7ec6be24871d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:48.204282 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b07ec-b021-47e6-a815-7ec6be24871d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:48.204759 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b07ec-b021-47e6-a815-7ec6be24871d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:48.205224 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b07ec-b021-47e6-a815-7ec6be24871d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:48.216247 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76b07ec-b021-47e6-a815-7ec6be24871d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:48.218128 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76b07ec-b021-47e6-a815-7ec6be24871d None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:48.230935 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-e76b07ec-b021-47e6-a815-7ec6be24871d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:48.230935 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:48.230935 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:48.230935 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:48.230935 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:48.230935 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:48.230935 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:48.230935 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:48.230935 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:48.230935 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:48.230935 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:48.230935 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:48.230935 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 05:06:48.230935 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 05:06:48.230935 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 05:06:48.231944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 05:06:48.231944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:48.231944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:48.231944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:48.231944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:48.231944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:48.231944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:48.231944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:48.231944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:48.231944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:48.231944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:48.231944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:48.231944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:48.231944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:48.231944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:48.231944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:48.231944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:48.233159 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 05:06:48.233159 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 05:06:48.233159 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 05:06:48.233159 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 05:06:48.233159 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 05:06:48.233159 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 05:06:48.233159 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:48.233159 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:48.236301 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 437/890] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:06:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:48.246029 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ae45e5-1a34-410e-b11f-29129c091195 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:48.287186 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76b07ec-b021-47e6-a815-7ec6be24871d None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:48.356673 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ae45e5-1a34-410e-b11f-29129c091195 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:48.389957 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76b07ec-b021-47e6-a815-7ec6be24871d None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:48.400522 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557f808-4281-4a76-8c8f-d1098ee27d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:48.401398 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557f808-4281-4a76-8c8f-d1098ee27d27 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:48.401674 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557f808-4281-4a76-8c8f-d1098ee27d27 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:48.401939 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557f808-4281-4a76-8c8f-d1098ee27d27 None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:48.402826 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8557f808-4281-4a76-8c8f-d1098ee27d27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:48.403155 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8557f808-4281-4a76-8c8f-d1098ee27d27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:48.405252 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8557f808-4281-4a76-8c8f-d1098ee27d27 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:48.418336 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:48.426222 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 454/891] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 05:06:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:48.427093 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557f808-4281-4a76-8c8f-d1098ee27d27 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:48.427310 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9d7bf3-44b7-4399-b303-b86d4f0e773e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:48.428144 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d7bf3-44b7-4399-b303-b86d4f0e773e None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:48.428535 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d7bf3-44b7-4399-b303-b86d4f0e773e None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:48.428897 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d7bf3-44b7-4399-b303-b86d4f0e773e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:48.429673 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9d7bf3-44b7-4399-b303-b86d4f0e773e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:48.429866 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:48.432062 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9d7bf3-44b7-4399-b303-b86d4f0e773e None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:48.442400 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-7b9d7bf3-44b7-4399-b303-b86d4f0e773e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 05:06:48.442400 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:48.442400 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 05:06:48.442400 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 05:06:48.442400 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 05:06:48.442400 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 05:06:48.442400 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 05:06:48.442400 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:06:48.442400 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 05:06:48.442400 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:48.442400 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 05:06:48.442400 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:48.442400 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:48.442400 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:48.442400 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:48.442400 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:48.442400 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:48.443626 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:48.443626 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:48.443626 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:48.443626 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:48.443626 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:48.443626 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:48.443626 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 05:06:48.443626 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 05:06:48.443626 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 05:06:48.443626 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 05:06:48.443626 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:48.443626 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:48.443626 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:48.443626 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:48.443626 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:48.443626 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:48.444691 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:48.444691 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:48.444691 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:48.444691 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:48.444691 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:48.444691 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:48.444691 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:48.444691 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:48.444691 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:48.444691 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:48.444691 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 05:06:48.444691 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:06:48.444691 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 05:06:48.444691 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:48.451232 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 438/892] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Aug 31 05:06:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:06:48.452603 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9d7bf3-44b7-4399-b303-b86d4f0e773e None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:48.554276 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9d7bf3-44b7-4399-b303-b86d4f0e773e None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:48.555641 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557f808-4281-4a76-8c8f-d1098ee27d27 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:48.596082 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef0a033-60bb-4d37-8e8f-8f6ac071ad48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:48.597101 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0a033-60bb-4d37-8e8f-8f6ac071ad48 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:48.597621 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0a033-60bb-4d37-8e8f-8f6ac071ad48 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:48.598175 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0a033-60bb-4d37-8e8f-8f6ac071ad48 None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:48.599131 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef0a033-60bb-4d37-8e8f-8f6ac071ad48 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:48.599590 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef0a033-60bb-4d37-8e8f-8f6ac071ad48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:48.599704 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dff503-f795-4a57-bd96-3d60e675c1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:48.600380 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dff503-f795-4a57-bd96-3d60e675c1b3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:48.600745 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dff503-f795-4a57-bd96-3d60e675c1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:48.601100 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dff503-f795-4a57-bd96-3d60e675c1b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:48.601731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef0a033-60bb-4d37-8e8f-8f6ac071ad48 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:48.602014 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dff503-f795-4a57-bd96-3d60e675c1b3 None admin] RBAC: Adding query filter params (role.id=f7c0c9eac46c42179e586dc4c1726356, scope.project.id=e01ce1b1282e44a39d659b87f120995e, user.id=09bf6db88ce8494a937211a814c9ff84) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:48.602419 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dff503-f795-4a57-bd96-3d60e675c1b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:48.603913 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dff503-f795-4a57-bd96-3d60e675c1b3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:48.608874 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 439/893] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Aug 31 05:06:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:06:48.618154 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 455/894] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:06:48 2022] GET /identity/v3/role_assignments?user.id=09bf6db88ce8494a937211a814c9ff84&scope.project.id=e01ce1b1282e44a39d659b87f120995e&role.id=f7c0c9eac46c42179e586dc4c1726356 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:48.620212 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef0a033-60bb-4d37-8e8f-8f6ac071ad48 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:48.671194 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:48.736536 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef0a033-60bb-4d37-8e8f-8f6ac071ad48 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:48.773751 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d31e9f-5c25-4f8b-bde1-60ba8af1ca98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:48.774403 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d31e9f-5c25-4f8b-bde1-60ba8af1ca98 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:48.774681 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d31e9f-5c25-4f8b-bde1-60ba8af1ca98 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:48.774953 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d31e9f-5c25-4f8b-bde1-60ba8af1ca98 None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:48.775609 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d31e9f-5c25-4f8b-bde1-60ba8af1ca98 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:48.775915 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d31e9f-5c25-4f8b-bde1-60ba8af1ca98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:48.777312 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d31e9f-5c25-4f8b-bde1-60ba8af1ca98 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:48.782946 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 440/895] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Aug 31 05:06:48 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:49.112195 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd21e67-abae-4717-b3b4-ee60a3c3e5fa None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:49.112726 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd21e67-abae-4717-b3b4-ee60a3c3e5fa None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:49.112726 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd21e67-abae-4717-b3b4-ee60a3c3e5fa None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:49.114347 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 456/896] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:49.119980 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a07a2-b1c1-4dea-b7a6-958db723b640 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:49.120265 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a07a2-b1c1-4dea-b7a6-958db723b640 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:49.120537 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a07a2-b1c1-4dea-b7a6-958db723b640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:49.139264 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-2e9a07a2-b1c1-4dea-b7a6-958db723b640 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:49.140458 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-2e9a07a2-b1c1-4dea-b7a6-958db723b640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:49.212457 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 441/897] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:49 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:06:49.478510 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd21e67-abae-4717-b3b4-ee60a3c3e5fa None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:49.548308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:49.584909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd21e67-abae-4717-b3b4-ee60a3c3e5fa None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:49.623172 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61210a94-b31e-4c45-ba9a-7b1a96a5fcb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:49.624173 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61210a94-b31e-4c45-ba9a-7b1a96a5fcb2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:49.624675 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61210a94-b31e-4c45-ba9a-7b1a96a5fcb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:49.625159 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61210a94-b31e-4c45-ba9a-7b1a96a5fcb2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:49.625965 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61210a94-b31e-4c45-ba9a-7b1a96a5fcb2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:49.627529 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61210a94-b31e-4c45-ba9a-7b1a96a5fcb2 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:49.637212 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-61210a94-b31e-4c45-ba9a-7b1a96a5fcb2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 05:06:49.637212 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:49.637212 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 05:06:49.637212 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 05:06:49.637212 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 05:06:49.637212 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 05:06:49.637212 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 05:06:49.637212 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:06:49.637212 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 05:06:49.637212 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:49.637212 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 05:06:49.637212 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:49.637212 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:49.637212 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:49.637212 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:49.637212 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:49.637212 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:49.638560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:49.638560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:49.638560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:49.638560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:49.638560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:49.638560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:49.638560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 05:06:49.638560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 05:06:49.638560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 05:06:49.638560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 05:06:49.638560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:49.638560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:49.638560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:49.638560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:49.638560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:49.638560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:49.639629 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:49.639629 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:49.639629 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:49.639629 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:49.639629 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:49.639629 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:49.639629 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:49.639629 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:49.639629 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:49.639629 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:49.639629 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 05:06:49.639629 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:06:49.639629 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 05:06:49.639629 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:49.642212 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 457/898] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 05:06:49 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:49.648912 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a07a2-b1c1-4dea-b7a6-958db723b640 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:49.684737 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c30f3-93c5-400c-80a0-df8c02fa937b None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:49.685064 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c30f3-93c5-400c-80a0-df8c02fa937b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:49.685273 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c30f3-93c5-400c-80a0-df8c02fa937b None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:49.687926 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 458/899] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 05:06:49.691730 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08bf2e7-7d4a-41c0-8945-ec0b6b1ee7bc None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:49.692006 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08bf2e7-7d4a-41c0-8945-ec0b6b1ee7bc None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:49.692277 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08bf2e7-7d4a-41c0-8945-ec0b6b1ee7bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:49.697133 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:49.711366 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d08bf2e7-7d4a-41c0-8945-ec0b6b1ee7bc None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:49.712541 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d08bf2e7-7d4a-41c0-8945-ec0b6b1ee7bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:49.771235 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a07a2-b1c1-4dea-b7a6-958db723b640 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:49.797273 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 459/900] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:49 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 05:06:49.813251 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659ba074-c110-4b38-8a5c-7ae6b1a7f90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:49.813984 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659ba074-c110-4b38-8a5c-7ae6b1a7f90f None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:49.814385 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659ba074-c110-4b38-8a5c-7ae6b1a7f90f None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:49.814753 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659ba074-c110-4b38-8a5c-7ae6b1a7f90f None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:49.815482 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659ba074-c110-4b38-8a5c-7ae6b1a7f90f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:49.815878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659ba074-c110-4b38-8a5c-7ae6b1a7f90f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:49.817361 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659ba074-c110-4b38-8a5c-7ae6b1a7f90f None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:49.822566 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 442/901] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 05:06:49 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:49.830218 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08bf2e7-7d4a-41c0-8945-ec0b6b1ee7bc None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:49.894870 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:49.936639 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08bf2e7-7d4a-41c0-8945-ec0b6b1ee7bc None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:49.970101 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a13b22-5efc-496b-8493-32c6ec1e78be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:49.970709 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a13b22-5efc-496b-8493-32c6ec1e78be None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:49.970989 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a13b22-5efc-496b-8493-32c6ec1e78be None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:49.971256 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a13b22-5efc-496b-8493-32c6ec1e78be None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:49.971881 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a13b22-5efc-496b-8493-32c6ec1e78be None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:49.972178 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a13b22-5efc-496b-8493-32c6ec1e78be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:49.973546 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a13b22-5efc-496b-8493-32c6ec1e78be None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:49.978696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 460/902] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 05:06:49 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:49.984838 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659ba074-c110-4b38-8a5c-7ae6b1a7f90f None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:50.073372 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a13b22-5efc-496b-8493-32c6ec1e78be None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:50.093204 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659ba074-c110-4b38-8a5c-7ae6b1a7f90f None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:50.110384 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:50.137447 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d9e6c2-0aa7-4388-92d4-7da8125f142b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:50.138104 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9e6c2-0aa7-4388-92d4-7da8125f142b None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:50.138384 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9e6c2-0aa7-4388-92d4-7da8125f142b None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:50.138651 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9e6c2-0aa7-4388-92d4-7da8125f142b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:50.139359 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d9e6c2-0aa7-4388-92d4-7da8125f142b None admin] RBAC: Adding query filter params (interface=public, service_id=565c3aa10b2a40ab9f6b5021157b7ac5, region_id=RegionOne) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:50.139808 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d9e6c2-0aa7-4388-92d4-7da8125f142b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:50.141766 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d9e6c2-0aa7-4388-92d4-7da8125f142b None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:50.152236 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 443/903] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Aug 31 05:06:49 2022] GET /identity/v3/endpoints?service_id=565c3aa10b2a40ab9f6b5021157b7ac5&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:50.154655 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d9e6c2-0aa7-4388-92d4-7da8125f142b None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:50.192674 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a13b22-5efc-496b-8493-32c6ec1e78be None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:50.224310 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:50.238317 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5266d69-95d6-428b-9912-24b30c5b7c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:50.238957 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5266d69-95d6-428b-9912-24b30c5b7c5e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:50.239234 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5266d69-95d6-428b-9912-24b30c5b7c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:50.239502 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5266d69-95d6-428b-9912-24b30c5b7c5e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:50.240087 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5266d69-95d6-428b-9912-24b30c5b7c5e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:50.241914 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5266d69-95d6-428b-9912-24b30c5b7c5e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:50.254180 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-e5266d69-95d6-428b-9912-24b30c5b7c5e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 05:06:50.254180 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:50.254180 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 05:06:50.254180 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 05:06:50.254180 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 05:06:50.254180 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 05:06:50.254180 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 05:06:50.254180 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:06:50.254180 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 05:06:50.254180 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:50.254180 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 05:06:50.254180 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:50.254180 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:50.254180 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:50.254180 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:50.254180 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:50.255198 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:50.255198 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:50.255198 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:50.255198 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:50.255198 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:50.255198 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:50.255198 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:50.255198 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 05:06:50.255198 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 05:06:50.255198 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 05:06:50.255198 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 05:06:50.255198 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:50.255198 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:50.255198 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:50.255198 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:50.255198 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:50.255198 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:50.261120 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:50.261120 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:50.261120 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:50.261120 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:50.261120 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:50.261120 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:50.261120 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:50.261120 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:50.261120 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:50.261120 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:50.261120 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 05:06:50.261120 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:06:50.261120 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 05:06:50.261120 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:50.261120 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 461/904] 10.222.0.36 () {56 vars in 1132 bytes} [Wed Aug 31 05:06:50 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:06:50.263548 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5266d69-95d6-428b-9912-24b30c5b7c5e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:50.270773 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d9e6c2-0aa7-4388-92d4-7da8125f142b None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:50.310032 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d773cd-c743-4eb0-ac40-27623c8cf364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:50.311050 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d773cd-c743-4eb0-ac40-27623c8cf364 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:50.311566 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d773cd-c743-4eb0-ac40-27623c8cf364 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:50.312024 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d773cd-c743-4eb0-ac40-27623c8cf364 None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:50.312915 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d773cd-c743-4eb0-ac40-27623c8cf364 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:50.314701 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d773cd-c743-4eb0-ac40-27623c8cf364 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:50.330223 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 444/905] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Aug 31 05:06:50 2022] GET /identity/v3/services => generated 1268 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 05:06:50.383263 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5266d69-95d6-428b-9912-24b30c5b7c5e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:50.420107 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8404a49c-a969-4354-86ac-84a0910b212f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:50.421229 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8404a49c-a969-4354-86ac-84a0910b212f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:50.421714 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8404a49c-a969-4354-86ac-84a0910b212f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:50.422208 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8404a49c-a969-4354-86ac-84a0910b212f None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:50.423043 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8404a49c-a969-4354-86ac-84a0910b212f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:50.423629 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8404a49c-a969-4354-86ac-84a0910b212f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:50.425333 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8404a49c-a969-4354-86ac-84a0910b212f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:50.434739 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 462/906] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Aug 31 05:06:50 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:06:50.437588 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d773cd-c743-4eb0-ac40-27623c8cf364 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:50.470744 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:50.547063 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d773cd-c743-4eb0-ac40-27623c8cf364 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:50.582340 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffe83da-7ba6-4515-a360-7a5b7945688a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:50.583070 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe83da-7ba6-4515-a360-7a5b7945688a None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:50.583436 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe83da-7ba6-4515-a360-7a5b7945688a None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:50.583797 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe83da-7ba6-4515-a360-7a5b7945688a None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:50.584517 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffe83da-7ba6-4515-a360-7a5b7945688a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:50.584910 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffe83da-7ba6-4515-a360-7a5b7945688a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:50.586382 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffe83da-7ba6-4515-a360-7a5b7945688a None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:50.595767 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 445/907] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Aug 31 05:06:50 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:50.597819 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8404a49c-a969-4354-86ac-84a0910b212f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:50.629821 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ca9ee-d5cf-4a23-b370-7a27f6586aaf None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:50.630182 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ca9ee-d5cf-4a23-b370-7a27f6586aaf None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:50.630381 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ca9ee-d5cf-4a23-b370-7a27f6586aaf None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:50.631996 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 446/908] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 05:06:50.637078 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1e597-2bef-46bb-aa86-3ddde5fa4447 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:50.640763 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1e597-2bef-46bb-aa86-3ddde5fa4447 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:50.640763 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1e597-2bef-46bb-aa86-3ddde5fa4447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:50.661317 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-27f1e597-2bef-46bb-aa86-3ddde5fa4447 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:50.662548 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-27f1e597-2bef-46bb-aa86-3ddde5fa4447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:50.672613 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:50.725843 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8404a49c-a969-4354-86ac-84a0910b212f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:50.745244 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 447/909] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:50 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 05:06:50.746927 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342890c-d81c-45cb-b98a-80094515a098 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:50.747421 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342890c-d81c-45cb-b98a-80094515a098 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:50.747865 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342890c-d81c-45cb-b98a-80094515a098 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:50.749370 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 448/910] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:50.755229 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259eb9c3-0114-4f91-a7b4-747368739050 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:50.755509 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259eb9c3-0114-4f91-a7b4-747368739050 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:50.755779 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259eb9c3-0114-4f91-a7b4-747368739050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:50.766900 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a7b649-d12b-4602-a8a8-a886286f4dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:50.767519 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a7b649-d12b-4602-a8a8-a886286f4dcb None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:50.767797 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a7b649-d12b-4602-a8a8-a886286f4dcb None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:50.768065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a7b649-d12b-4602-a8a8-a886286f4dcb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:50.768773 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a7b649-d12b-4602-a8a8-a886286f4dcb None admin] RBAC: Adding query filter params (interface=public, service_id=d1aaa4126d1349398bbb9e0f501a8779, region_id=RegionOne) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:50.769086 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a7b649-d12b-4602-a8a8-a886286f4dcb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:50.770502 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a7b649-d12b-4602-a8a8-a886286f4dcb None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:50.776823 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-259eb9c3-0114-4f91-a7b4-747368739050 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:50.778526 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-259eb9c3-0114-4f91-a7b4-747368739050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:50.778656 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 463/911] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Aug 31 05:06:50 2022] GET /identity/v3/endpoints?service_id=d1aaa4126d1349398bbb9e0f501a8779&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:50.786120 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a7b649-d12b-4602-a8a8-a886286f4dcb None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:50.810468 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:50.853892 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 449/912] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:50 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:06:50.901522 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a7b649-d12b-4602-a8a8-a886286f4dcb None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:50.935211 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc3837-fdaf-4454-93a6-381b621067e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:50.936113 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc3837-fdaf-4454-93a6-381b621067e2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:50.936524 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc3837-fdaf-4454-93a6-381b621067e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:50.936935 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc3837-fdaf-4454-93a6-381b621067e2 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:50.937669 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dc3837-fdaf-4454-93a6-381b621067e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:50.939223 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dc3837-fdaf-4454-93a6-381b621067e2 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:50.946361 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:50.950157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 464/913] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Aug 31 05:06:50 2022] GET /identity/v3/services => generated 1268 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 05:06:51.030281 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259eb9c3-0114-4f91-a7b4-747368739050 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:51.135926 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc3837-fdaf-4454-93a6-381b621067e2 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:51.140931 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259eb9c3-0114-4f91-a7b4-747368739050 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:51.185729 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a652e-1a9b-47b5-9b0e-832b63efe3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:51.187132 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a652e-1a9b-47b5-9b0e-832b63efe3fe None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:51.187613 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a652e-1a9b-47b5-9b0e-832b63efe3fe None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:51.188181 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a652e-1a9b-47b5-9b0e-832b63efe3fe None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:51.199685 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a652e-1a9b-47b5-9b0e-832b63efe3fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:06:51.205733 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-d94a652e-1a9b-47b5-9b0e-832b63efe3fe None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 05:06:51.205733 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:51.205733 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:51.205733 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:51.205733 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:51.205733 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:51.205733 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:51.205733 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:51.205733 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:51.205733 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:51.205733 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:51.205733 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:51.205733 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:06:51.205733 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:06:51.205733 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:06:51.205733 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:06:51.206703 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:06:51.206703 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:06:51.206703 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:06:51.206703 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:06:51.206703 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:51.206703 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:51.206703 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:51.206703 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:51.206703 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:51.206703 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:51.206703 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:51.206703 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:51.206703 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:51.206703 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:51.206703 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:51.206703 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:51.207557 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:51.207557 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:51.207557 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:51.207557 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:51.207557 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:06:51.207557 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:06:51.207557 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:06:51.207557 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:06:51.207557 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:06:51.207557 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:06:51.207557 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 05:06:51.207557 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:51.210329 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 450/914] 10.222.0.36 () {56 vars in 1120 bytes} [Wed Aug 31 05:06:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:51.218276 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a652e-1a9b-47b5-9b0e-832b63efe3fe None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:51.237737 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:51.253679 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc3837-fdaf-4454-93a6-381b621067e2 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:51.295247 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbea4dc-77ac-4b55-ab35-10fbed78c480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:51.296418 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbea4dc-77ac-4b55-ab35-10fbed78c480 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:51.296832 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbea4dc-77ac-4b55-ab35-10fbed78c480 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:51.297390 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbea4dc-77ac-4b55-ab35-10fbed78c480 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:51.298667 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbea4dc-77ac-4b55-ab35-10fbed78c480 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:51.299891 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbea4dc-77ac-4b55-ab35-10fbed78c480 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:51.322354 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 465/915] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Aug 31 05:06:51 2022] POST /identity/v3/services => generated 241 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:06:51.335362 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a652e-1a9b-47b5-9b0e-832b63efe3fe None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:51.381345 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28bca05-fd8f-4c68-95a0-be87ce6ce5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:51.382284 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28bca05-fd8f-4c68-95a0-be87ce6ce5d8 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:51.382697 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28bca05-fd8f-4c68-95a0-be87ce6ce5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:51.383102 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28bca05-fd8f-4c68-95a0-be87ce6ce5d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:51.383911 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28bca05-fd8f-4c68-95a0-be87ce6ce5d8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:51.384340 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28bca05-fd8f-4c68-95a0-be87ce6ce5d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:51.386032 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28bca05-fd8f-4c68-95a0-be87ce6ce5d8 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:51.396351 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 451/916] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Aug 31 05:06:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:51.404948 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbea4dc-77ac-4b55-ab35-10fbed78c480 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:51.456105 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:51.524654 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbea4dc-77ac-4b55-ab35-10fbed78c480 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:51.557792 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac63f1a-df10-421e-9d5e-11b7aece266e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:51.558451 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac63f1a-df10-421e-9d5e-11b7aece266e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:51.558723 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac63f1a-df10-421e-9d5e-11b7aece266e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:51.558982 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac63f1a-df10-421e-9d5e-11b7aece266e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:51.570080 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac63f1a-df10-421e-9d5e-11b7aece266e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:51.571509 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac63f1a-df10-421e-9d5e-11b7aece266e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:51.586508 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-bac63f1a-df10-421e-9d5e-11b7aece266e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:51.586508 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:51.586508 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:51.586508 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:51.586508 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:51.586508 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:51.586508 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:51.586508 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:51.586508 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:51.586508 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:51.586508 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:51.586508 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:51.586508 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:51.586508 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:51.586508 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:51.586508 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:51.587670 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:51.587670 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:51.587670 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:51.587670 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:51.587670 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:51.587670 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:51.587670 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:51.587670 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:51.587670 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:51.587670 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:51.587670 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:51.587670 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:51.587670 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:51.587670 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:51.587670 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:51.587670 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:51.587670 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:51.588816 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:51.588816 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:51.588816 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:51.588816 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 466/917] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:51.594356 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28bca05-fd8f-4c68-95a0-be87ce6ce5d8 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:51.670471 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:51.701751 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28bca05-fd8f-4c68-95a0-be87ce6ce5d8 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:51.738250 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64026ddd-c961-4540-a060-7514e1fc88e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:51.738858 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64026ddd-c961-4540-a060-7514e1fc88e9 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:51.739136 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64026ddd-c961-4540-a060-7514e1fc88e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:51.739404 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64026ddd-c961-4540-a060-7514e1fc88e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:51.740172 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64026ddd-c961-4540-a060-7514e1fc88e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:51.740533 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64026ddd-c961-4540-a060-7514e1fc88e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:51.741952 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64026ddd-c961-4540-a060-7514e1fc88e9 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:51.755000 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 452/918] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:51.760776 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac63f1a-df10-421e-9d5e-11b7aece266e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:51.860964 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:51.871305 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac63f1a-df10-421e-9d5e-11b7aece266e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:51.908943 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a42b7-00ad-4091-bbbe-d9d8f30459e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:51.909921 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a42b7-00ad-4091-bbbe-d9d8f30459e5 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:51.910474 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a42b7-00ad-4091-bbbe-d9d8f30459e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:51.910937 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a42b7-00ad-4091-bbbe-d9d8f30459e5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:51.919391 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296a42b7-00ad-4091-bbbe-d9d8f30459e5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:51.921375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296a42b7-00ad-4091-bbbe-d9d8f30459e5 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:51.933687 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-296a42b7-00ad-4091-bbbe-d9d8f30459e5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 05:06:51.933687 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:51.933687 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:51.933687 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:51.933687 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:51.933687 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:51.933687 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:51.933687 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:51.933687 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:51.933687 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:51.933687 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:51.933687 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:51.933687 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 05:06:51.933687 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 05:06:51.933687 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 05:06:51.933687 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 05:06:51.934748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:51.934748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:51.934748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 05:06:51.934748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:51.934748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 05:06:51.934748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:51.934748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:51.934748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:51.934748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:51.934748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:51.934748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:51.934748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:51.934748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:51.934748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:51.934748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:51.934748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:51.934748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:51.934748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:51.935713 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:51.935713 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:51.935713 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 05:06:51.935713 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 05:06:51.935713 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 05:06:51.935713 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 05:06:51.935713 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 05:06:51.935713 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 05:06:51.935713 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 05:06:51.935713 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:51.938316 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 467/919] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Aug 31 05:06:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:51.946153 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64026ddd-c961-4540-a060-7514e1fc88e9 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:52.050361 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64026ddd-c961-4540-a060-7514e1fc88e9 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:52.080077 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:52.086649 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee50273-c0b8-4abf-b2e8-a3f32212231f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:52.087281 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee50273-c0b8-4abf-b2e8-a3f32212231f None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:52.087718 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee50273-c0b8-4abf-b2e8-a3f32212231f None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:52.087987 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee50273-c0b8-4abf-b2e8-a3f32212231f None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:52.088772 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee50273-c0b8-4abf-b2e8-a3f32212231f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:52.089135 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee50273-c0b8-4abf-b2e8-a3f32212231f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:52.090674 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee50273-c0b8-4abf-b2e8-a3f32212231f None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:52.111327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 453/920] 10.222.0.36 () {56 vars in 1149 bytes} [Wed Aug 31 05:06:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:52.116603 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a42b7-00ad-4091-bbbe-d9d8f30459e5 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:52.218623 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a42b7-00ad-4091-bbbe-d9d8f30459e5 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:52.252881 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc97a3c-523f-40af-a2b7-bcc75d35665a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:52.253517 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc97a3c-523f-40af-a2b7-bcc75d35665a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:52.253794 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc97a3c-523f-40af-a2b7-bcc75d35665a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:52.254352 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc97a3c-523f-40af-a2b7-bcc75d35665a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:52.264763 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc97a3c-523f-40af-a2b7-bcc75d35665a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:52.266228 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc97a3c-523f-40af-a2b7-bcc75d35665a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:52.280258 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-9cc97a3c-523f-40af-a2b7-bcc75d35665a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:52.280258 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:52.280258 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:52.280258 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:52.280258 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:52.280258 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:52.280258 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:52.280258 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:52.280258 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:52.280258 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:52.280258 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:52.280258 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:52.280258 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:52.280258 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:52.280258 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:52.280258 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:52.281190 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:52.281190 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:52.281190 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:52.281190 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:52.281190 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:52.281190 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:52.281190 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:52.281190 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:52.281190 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:52.281190 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:52.281190 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:52.281190 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:52.281190 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:52.281190 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:52.281190 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:52.281190 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:52.281190 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:52.282111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:52.282111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:52.282111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:52.282111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 468/921] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:52.287598 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee50273-c0b8-4abf-b2e8-a3f32212231f None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:52.313906 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033751f8-a289-4f52-9336-9dde7ffe73c8 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:52.314235 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033751f8-a289-4f52-9336-9dde7ffe73c8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:52.314490 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033751f8-a289-4f52-9336-9dde7ffe73c8 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:52.315972 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 469/922] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 05:06:52.321229 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b459dae6-f0e2-41f3-90b5-d433aba0ba30 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:52.321509 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b459dae6-f0e2-41f3-90b5-d433aba0ba30 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:52.321778 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b459dae6-f0e2-41f3-90b5-d433aba0ba30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:52.329718 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:52.340474 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b459dae6-f0e2-41f3-90b5-d433aba0ba30 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:52.341700 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b459dae6-f0e2-41f3-90b5-d433aba0ba30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:52.398125 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee50273-c0b8-4abf-b2e8-a3f32212231f None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:52.422040 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 470/923] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 05:06:52.438613 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c1e890-329f-42ee-b546-558476a8db58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:52.439228 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c1e890-329f-42ee-b546-558476a8db58 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:52.439504 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c1e890-329f-42ee-b546-558476a8db58 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:52.439773 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c1e890-329f-42ee-b546-558476a8db58 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:52.440426 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c1e890-329f-42ee-b546-558476a8db58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:52.440725 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c1e890-329f-42ee-b546-558476a8db58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:52.442184 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c1e890-329f-42ee-b546-558476a8db58 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:52.454309 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 454/924] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:52.463773 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b459dae6-f0e2-41f3-90b5-d433aba0ba30 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:52.571417 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b459dae6-f0e2-41f3-90b5-d433aba0ba30 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:52.607297 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd68c96-4ef1-49d4-9ce4-53f788c586c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:52.607931 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd68c96-4ef1-49d4-9ce4-53f788c586c4 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:52.608256 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd68c96-4ef1-49d4-9ce4-53f788c586c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:52.608528 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd68c96-4ef1-49d4-9ce4-53f788c586c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:52.610478 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:52.615602 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd68c96-4ef1-49d4-9ce4-53f788c586c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:52.617102 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd68c96-4ef1-49d4-9ce4-53f788c586c4 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:52.629245 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-5dd68c96-4ef1-49d4-9ce4-53f788c586c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:52.629245 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:52.629245 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:52.629245 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:52.629245 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:52.629245 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:52.629245 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:52.629245 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:52.629245 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:52.629245 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:52.629245 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:52.629245 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:52.629245 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 05:06:52.629245 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 05:06:52.629245 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 05:06:52.630166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 05:06:52.630166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:52.630166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:52.630166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:52.630166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:52.630166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:52.630166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:52.630166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:52.630166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:52.630166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:52.630166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:52.630166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:52.630166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:52.630166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:52.630166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:52.630166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:52.630166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:52.631086 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 05:06:52.631086 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 05:06:52.631086 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 05:06:52.631086 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 05:06:52.631086 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 05:06:52.631086 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 05:06:52.631086 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:52.631086 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:52.634333 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 471/925] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:06:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:52.636714 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c1e890-329f-42ee-b546-558476a8db58 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:52.689708 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:52.696672 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd68c96-4ef1-49d4-9ce4-53f788c586c4 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:52.758770 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c1e890-329f-42ee-b546-558476a8db58 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:52.801137 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a6280f-81a2-440f-a535-d3317471c5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:52.802653 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6280f-81a2-440f-a535-d3317471c5ea None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:52.803247 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6280f-81a2-440f-a535-d3317471c5ea None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:52.803673 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6280f-81a2-440f-a535-d3317471c5ea None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:52.804944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a6280f-81a2-440f-a535-d3317471c5ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:52.805658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a6280f-81a2-440f-a535-d3317471c5ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:52.808045 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a6280f-81a2-440f-a535-d3317471c5ea None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:52.815189 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd68c96-4ef1-49d4-9ce4-53f788c586c4 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:52.827091 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 455/926] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 05:06:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:52.834745 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a6280f-81a2-440f-a535-d3317471c5ea None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:52.860134 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b4be49-5693-44aa-9274-5cdacc6d8c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:52.861332 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4be49-5693-44aa-9274-5cdacc6d8c36 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:52.862048 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4be49-5693-44aa-9274-5cdacc6d8c36 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:52.862502 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4be49-5693-44aa-9274-5cdacc6d8c36 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:52.863666 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b4be49-5693-44aa-9274-5cdacc6d8c36 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:52.865720 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b4be49-5693-44aa-9274-5cdacc6d8c36 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:52.878226 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-77b4be49-5693-44aa-9274-5cdacc6d8c36 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 05:06:52.878226 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:52.878226 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 05:06:52.878226 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 05:06:52.878226 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 05:06:52.878226 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 05:06:52.878226 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 05:06:52.878226 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:06:52.878226 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 05:06:52.878226 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:52.878226 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 05:06:52.878226 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:52.878226 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:52.878226 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:52.878226 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:52.878226 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:52.878226 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:52.879243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:52.879243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:52.879243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:52.879243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:52.879243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:52.879243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:52.879243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 05:06:52.879243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 05:06:52.879243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 05:06:52.879243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 05:06:52.879243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:52.879243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:52.879243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:52.879243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:52.879243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:52.879243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:52.880100 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:52.880100 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:52.880100 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:52.880100 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:52.880100 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:52.880100 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:52.880100 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:52.880100 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:52.880100 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:52.880100 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:52.880100 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 05:06:52.880100 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:06:52.880100 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 05:06:52.880100 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:52.880100 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 472/927] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 05:06:52 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:06:52.887194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b4be49-5693-44aa-9274-5cdacc6d8c36 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:52.957340 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a6280f-81a2-440f-a535-d3317471c5ea None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:52.960481 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:52.975376 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:53.001360 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3342179-ceac-406e-9489-c5b56e30cbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:53.002012 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3342179-ceac-406e-9489-c5b56e30cbf7 None admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:53.002519 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3342179-ceac-406e-9489-c5b56e30cbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:53.002639 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3342179-ceac-406e-9489-c5b56e30cbf7 None admin] PATH_INFO: `/v3/projects/e01ce1b1282e44a39d659b87f120995e/users/09bf6db88ce8494a937211a814c9ff84/roles/f7c0c9eac46c42179e586dc4c1726356` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:53.004786 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3342179-ceac-406e-9489-c5b56e30cbf7 None admin] RBAC: Authorizing `identity:create_grant(project_id=e01ce1b1282e44a39d659b87f120995e, user_id=09bf6db88ce8494a937211a814c9ff84, role_id=f7c0c9eac46c42179e586dc4c1726356)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:53.006178 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3342179-ceac-406e-9489-c5b56e30cbf7 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:53.008607 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b4be49-5693-44aa-9274-5cdacc6d8c36 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:53.029748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 456/928] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 05:06:52 2022] PUT /identity/v3/projects/e01ce1b1282e44a39d659b87f120995e/users/09bf6db88ce8494a937211a814c9ff84/roles/f7c0c9eac46c42179e586dc4c1726356 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:06:53.029748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8517dd-a113-4aa7-9887-81adf4d93879 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:53.029748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8517dd-a113-4aa7-9887-81adf4d93879 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:53.029748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8517dd-a113-4aa7-9887-81adf4d93879 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:53.029748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 457/929] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:53.031708 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0e383-d322-453f-bbc5-b5968f2e8c62 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:53.031992 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0e383-d322-453f-bbc5-b5968f2e8c62 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:53.032262 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0e383-d322-453f-bbc5-b5968f2e8c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:53.051519 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-d9c0e383-d322-453f-bbc5-b5968f2e8c62 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:53.052740 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-d9c0e383-d322-453f-bbc5-b5968f2e8c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:53.057555 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f56f18-b52f-49d5-9cb5-801d112e6520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:53.058572 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f56f18-b52f-49d5-9cb5-801d112e6520 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:53.059105 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f56f18-b52f-49d5-9cb5-801d112e6520 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:53.059664 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f56f18-b52f-49d5-9cb5-801d112e6520 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:53.060507 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f56f18-b52f-49d5-9cb5-801d112e6520 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:53.061103 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f56f18-b52f-49d5-9cb5-801d112e6520 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:53.062842 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f56f18-b52f-49d5-9cb5-801d112e6520 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:53.069432 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 473/930] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 05:06:52 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:53.075002 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f56f18-b52f-49d5-9cb5-801d112e6520 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:53.138126 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 458/931] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:06:53.196520 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f56f18-b52f-49d5-9cb5-801d112e6520 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:53.221608 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:53.235183 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be1e546-b9ea-49b8-a4e4-3d0075bc7877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:53.235788 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be1e546-b9ea-49b8-a4e4-3d0075bc7877 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:53.236061 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be1e546-b9ea-49b8-a4e4-3d0075bc7877 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:53.236323 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be1e546-b9ea-49b8-a4e4-3d0075bc7877 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:53.236942 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be1e546-b9ea-49b8-a4e4-3d0075bc7877 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:53.237238 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be1e546-b9ea-49b8-a4e4-3d0075bc7877 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:53.238670 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be1e546-b9ea-49b8-a4e4-3d0075bc7877 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:53.245022 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 474/932] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Aug 31 05:06:53 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:06:53.252513 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c0e383-d322-453f-bbc5-b5968f2e8c62 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:53.354799 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:53.357670 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c0e383-d322-453f-bbc5-b5968f2e8c62 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:53.394300 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b135f080-87d4-401e-9860-7f1e711c1b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:53.394815 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b135f080-87d4-401e-9860-7f1e711c1b37 None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:53.395089 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b135f080-87d4-401e-9860-7f1e711c1b37 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:53.395360 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b135f080-87d4-401e-9860-7f1e711c1b37 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:53.396047 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b135f080-87d4-401e-9860-7f1e711c1b37 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:53.397425 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b135f080-87d4-401e-9860-7f1e711c1b37 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:53.417457 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce7910d-c0b1-491d-a272-e0544d97d166 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:53.417963 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce7910d-c0b1-491d-a272-e0544d97d166 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:53.418101 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce7910d-c0b1-491d-a272-e0544d97d166 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:53.419716 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 475/933] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:53.420421 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 459/934] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:06:53 2022] POST /identity/v3/endpoints => generated 346 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:06:53.427985 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca88920-af32-49b8-81ad-b8a8a0dce234 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:53.428293 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca88920-af32-49b8-81ad-b8a8a0dce234 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:53.428595 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca88920-af32-49b8-81ad-b8a8a0dce234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:53.441201 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b135f080-87d4-401e-9860-7f1e711c1b37 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:53.449506 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-9ca88920-af32-49b8-81ad-b8a8a0dce234 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:53.450907 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9ca88920-af32-49b8-81ad-b8a8a0dce234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:53.547423 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 476/935] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:06:53.582350 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b135f080-87d4-401e-9860-7f1e711c1b37 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:53.619269 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50feb924-de56-4a76-ad70-266a244726e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:53.619909 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50feb924-de56-4a76-ad70-266a244726e5 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:53.620186 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50feb924-de56-4a76-ad70-266a244726e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:53.620457 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50feb924-de56-4a76-ad70-266a244726e5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:53.621048 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50feb924-de56-4a76-ad70-266a244726e5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:53.622572 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50feb924-de56-4a76-ad70-266a244726e5 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:53.631806 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-50feb924-de56-4a76-ad70-266a244726e5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 05:06:53.631806 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:53.631806 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 05:06:53.631806 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 05:06:53.631806 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 05:06:53.631806 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 05:06:53.631806 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 05:06:53.631806 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:06:53.631806 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 05:06:53.631806 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:53.631806 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 05:06:53.631806 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:53.631806 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:53.631806 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:53.631806 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:53.631806 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:53.632765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:53.632765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:53.632765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:53.632765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:53.632765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:53.632765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:53.632765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:53.632765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 05:06:53.632765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 05:06:53.632765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 05:06:53.632765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 05:06:53.632765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:53.632765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:53.632765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:53.632765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:53.632765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:53.632765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:53.634797 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:53.634797 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:53.634797 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:53.634797 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:53.634797 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:53.634797 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:53.634797 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:53.634797 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:53.634797 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:53.634797 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:53.634797 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 05:06:53.634797 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:06:53.634797 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 05:06:53.634797 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:53.634797 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 460/936] 10.222.0.36 () {56 vars in 1132 bytes} [Wed Aug 31 05:06:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:53.639401 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca88920-af32-49b8-81ad-b8a8a0dce234 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:53.748591 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca88920-af32-49b8-81ad-b8a8a0dce234 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:53.763473 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:53.785239 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72efe0c-d678-46d1-8400-582590a8ad70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:53.785844 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72efe0c-d678-46d1-8400-582590a8ad70 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:53.786141 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72efe0c-d678-46d1-8400-582590a8ad70 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:53.786684 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72efe0c-d678-46d1-8400-582590a8ad70 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:53.787303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72efe0c-d678-46d1-8400-582590a8ad70 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:53.787596 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72efe0c-d678-46d1-8400-582590a8ad70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:53.788952 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72efe0c-d678-46d1-8400-582590a8ad70 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:53.794119 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 477/937] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Aug 31 05:06:53 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:53.799584 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50feb924-de56-4a76-ad70-266a244726e5 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:53.823434 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:53.843559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72efe0c-d678-46d1-8400-582590a8ad70 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:53.912998 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50feb924-de56-4a76-ad70-266a244726e5 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:53.948652 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72efe0c-d678-46d1-8400-582590a8ad70 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:53.957312 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f41d843-5e63-4a45-9f39-06cc88c94336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:53.957980 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f41d843-5e63-4a45-9f39-06cc88c94336 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:53.958291 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f41d843-5e63-4a45-9f39-06cc88c94336 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:53.959020 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f41d843-5e63-4a45-9f39-06cc88c94336 None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:53.959668 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f41d843-5e63-4a45-9f39-06cc88c94336 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:53.960366 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f41d843-5e63-4a45-9f39-06cc88c94336 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:53.962036 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f41d843-5e63-4a45-9f39-06cc88c94336 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:53.973794 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 461/938] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Aug 31 05:06:53 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:53.974777 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f41d843-5e63-4a45-9f39-06cc88c94336 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:53.989755 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9443fbc-da23-4d40-a066-8a98b07ba229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:53.990430 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9443fbc-da23-4d40-a066-8a98b07ba229 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:53.990701 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9443fbc-da23-4d40-a066-8a98b07ba229 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:53.990962 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9443fbc-da23-4d40-a066-8a98b07ba229 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:53.991536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9443fbc-da23-4d40-a066-8a98b07ba229 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:53.993204 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9443fbc-da23-4d40-a066-8a98b07ba229 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:54.002463 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-c9443fbc-da23-4d40-a066-8a98b07ba229 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 05:06:54.002463 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:54.002463 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 05:06:54.002463 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 05:06:54.002463 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 05:06:54.002463 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 05:06:54.002463 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 05:06:54.002463 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:06:54.002463 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 05:06:54.002463 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:54.002463 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 05:06:54.002463 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:54.002463 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:54.002463 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:54.002463 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:54.002463 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:54.002463 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:54.003494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:54.003494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:54.003494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:54.003494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:54.003494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:54.003494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:54.003494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 05:06:54.003494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 05:06:54.003494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 05:06:54.003494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 05:06:54.003494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:54.003494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:54.003494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:54.003494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:54.003494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:54.003494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:54.004349 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:54.004349 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:54.004349 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:54.004349 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:54.004349 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:54.004349 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:54.004349 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:54.004349 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:54.004349 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:54.004349 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:54.004349 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 05:06:54.004349 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:06:54.004349 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 05:06:54.004349 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:54.008785 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 478/939] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Aug 31 05:06:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:54.009616 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9443fbc-da23-4d40-a066-8a98b07ba229 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:54.056181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:54.084082 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:54.102905 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f41d843-5e63-4a45-9f39-06cc88c94336 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:54.135266 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9443fbc-da23-4d40-a066-8a98b07ba229 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:54.152350 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40702f69-eed4-45d9-a513-3a8c0bee4d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:54.153815 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40702f69-eed4-45d9-a513-3a8c0bee4d10 None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:54.154274 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40702f69-eed4-45d9-a513-3a8c0bee4d10 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:54.154689 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40702f69-eed4-45d9-a513-3a8c0bee4d10 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:54.155524 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40702f69-eed4-45d9-a513-3a8c0bee4d10 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:54.157591 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40702f69-eed4-45d9-a513-3a8c0bee4d10 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:54.172080 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325bdc14-671b-4543-ab1b-4cef1bb2736d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:54.172860 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325bdc14-671b-4543-ab1b-4cef1bb2736d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:54.173240 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325bdc14-671b-4543-ab1b-4cef1bb2736d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:54.173725 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325bdc14-671b-4543-ab1b-4cef1bb2736d None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:54.174550 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325bdc14-671b-4543-ab1b-4cef1bb2736d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:54.174910 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325bdc14-671b-4543-ab1b-4cef1bb2736d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:54.177184 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325bdc14-671b-4543-ab1b-4cef1bb2736d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:54.192162 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 479/941] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Aug 31 05:06:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:54.192162 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325bdc14-671b-4543-ab1b-4cef1bb2736d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:54.192583 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 462/941] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:06:53 2022] POST /identity/v3/endpoints => generated 347 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:06:54.312976 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325bdc14-671b-4543-ab1b-4cef1bb2736d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:54.315983 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:54.357862 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8111965-0f00-4138-a703-00ac4a508ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:54.358745 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8111965-0f00-4138-a703-00ac4a508ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:54.359168 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8111965-0f00-4138-a703-00ac4a508ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:54.359604 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8111965-0f00-4138-a703-00ac4a508ac5 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:54.360369 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8111965-0f00-4138-a703-00ac4a508ac5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:54.360808 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8111965-0f00-4138-a703-00ac4a508ac5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:54.362344 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8111965-0f00-4138-a703-00ac4a508ac5 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:54.367432 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 480/942] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Aug 31 05:06:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:54.374635 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40702f69-eed4-45d9-a513-3a8c0bee4d10 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:54.488393 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40702f69-eed4-45d9-a513-3a8c0bee4d10 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:54.524270 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dfd0a4-e72a-4fcb-a461-e04f1a471025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:54.525335 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dfd0a4-e72a-4fcb-a461-e04f1a471025 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:54.525873 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dfd0a4-e72a-4fcb-a461-e04f1a471025 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:54.526414 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dfd0a4-e72a-4fcb-a461-e04f1a471025 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:54.527389 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dfd0a4-e72a-4fcb-a461-e04f1a471025 None admin] RBAC: Adding query filter params (interface=public, service_id=64bf473e5bec4e8090afac8ce11b3add, region_id=RegionOne) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:54.527958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dfd0a4-e72a-4fcb-a461-e04f1a471025 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:54.529549 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dfd0a4-e72a-4fcb-a461-e04f1a471025 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:54.535412 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 463/943] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Aug 31 05:06:54 2022] GET /identity/v3/endpoints?service_id=64bf473e5bec4e8090afac8ce11b3add&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:54.542038 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8111965-0f00-4138-a703-00ac4a508ac5 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:54.645760 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8111965-0f00-4138-a703-00ac4a508ac5 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:54.680420 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a0f32f-6483-4d69-aa89-a4698817fc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:54.681036 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0f32f-6483-4d69-aa89-a4698817fc5e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:54.681312 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0f32f-6483-4d69-aa89-a4698817fc5e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:54.681580 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0f32f-6483-4d69-aa89-a4698817fc5e None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:54.682210 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a0f32f-6483-4d69-aa89-a4698817fc5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:54.683584 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a0f32f-6483-4d69-aa89-a4698817fc5e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:54.693982 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 481/944] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Aug 31 05:06:54 2022] GET /identity/v3/services => generated 1497 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:06:55.074762 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe749a-8621-4b35-9afc-1d3f0215ad23 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:55.075827 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe749a-8621-4b35-9afc-1d3f0215ad23 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:55.076333 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe749a-8621-4b35-9afc-1d3f0215ad23 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:55.078248 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 464/945] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:55.083746 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549da3f-8542-417d-8d3b-ded6db397671 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:55.084306 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549da3f-8542-417d-8d3b-ded6db397671 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:55.084795 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549da3f-8542-417d-8d3b-ded6db397671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:55.103923 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-7549da3f-8542-417d-8d3b-ded6db397671 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:55.105455 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7549da3f-8542-417d-8d3b-ded6db397671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:55.129982 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:55.175545 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 482/946] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:06:55.392379 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47079b7-b39c-44f3-9cea-9bd97ebd9e0d None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:55.392555 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47079b7-b39c-44f3-9cea-9bd97ebd9e0d None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:55.392840 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47079b7-b39c-44f3-9cea-9bd97ebd9e0d None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:55.394463 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 465/947] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:55.399883 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792d58f-a9f9-4599-922f-a7e44ccf5e02 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:55.400168 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792d58f-a9f9-4599-922f-a7e44ccf5e02 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:55.400441 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792d58f-a9f9-4599-922f-a7e44ccf5e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:55.418492 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-1792d58f-a9f9-4599-922f-a7e44ccf5e02 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:55.419673 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1792d58f-a9f9-4599-922f-a7e44ccf5e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:55.446881 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47079b7-b39c-44f3-9cea-9bd97ebd9e0d None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:55.451539 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:55.491397 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 483/948] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:06:55.555080 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47079b7-b39c-44f3-9cea-9bd97ebd9e0d None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:55.589433 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4047941b-a8f0-4899-aa85-547cb66b3475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:55.590514 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4047941b-a8f0-4899-aa85-547cb66b3475 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:55.590996 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4047941b-a8f0-4899-aa85-547cb66b3475 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:55.591495 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4047941b-a8f0-4899-aa85-547cb66b3475 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:55.602769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4047941b-a8f0-4899-aa85-547cb66b3475 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:06:55.607803 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-4047941b-a8f0-4899-aa85-547cb66b3475 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 05:06:55.607803 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:55.607803 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:55.607803 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:55.607803 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:55.607803 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:55.607803 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:55.607803 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:55.607803 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:55.607803 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:55.607803 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:55.607803 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:55.607803 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:06:55.607803 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:06:55.607803 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:06:55.607803 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:06:55.608829 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:06:55.608829 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:06:55.608829 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:06:55.608829 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:06:55.608829 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:55.608829 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:55.608829 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:55.608829 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:55.608829 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:55.608829 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:55.608829 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:55.608829 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:55.608829 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:55.608829 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:55.608829 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:55.608829 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:55.609783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:55.609783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:55.609783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:55.609783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:55.609783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:06:55.609783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:06:55.609783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:06:55.609783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:06:55.609783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:06:55.609783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:06:55.609783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 05:06:55.609783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:55.612451 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 466/949] 10.222.0.36 () {56 vars in 1120 bytes} [Wed Aug 31 05:06:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:55.618971 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1792d58f-a9f9-4599-922f-a7e44ccf5e02 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:55.709416 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:55.721518 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1792d58f-a9f9-4599-922f-a7e44ccf5e02 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:55.756705 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5564b9-ec5f-4cf7-abb7-cad172f30f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:55.757365 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5564b9-ec5f-4cf7-abb7-cad172f30f05 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:55.757646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5564b9-ec5f-4cf7-abb7-cad172f30f05 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:55.757914 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5564b9-ec5f-4cf7-abb7-cad172f30f05 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:55.758612 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5564b9-ec5f-4cf7-abb7-cad172f30f05 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:55.758911 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5564b9-ec5f-4cf7-abb7-cad172f30f05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:55.760289 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5564b9-ec5f-4cf7-abb7-cad172f30f05 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:55.769187 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 484/950] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Aug 31 05:06:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:55.774493 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4047941b-a8f0-4899-aa85-547cb66b3475 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:55.781914 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5564b9-ec5f-4cf7-abb7-cad172f30f05 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:55.841860 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:55.886698 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4047941b-a8f0-4899-aa85-547cb66b3475 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:55.918288 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5564b9-ec5f-4cf7-abb7-cad172f30f05 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:55.930941 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41be2a7-877b-4c8a-b409-a6b9a44da1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:55.931575 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41be2a7-877b-4c8a-b409-a6b9a44da1b3 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:55.932165 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41be2a7-877b-4c8a-b409-a6b9a44da1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:55.932443 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41be2a7-877b-4c8a-b409-a6b9a44da1b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:55.946122 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41be2a7-877b-4c8a-b409-a6b9a44da1b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:55.948327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41be2a7-877b-4c8a-b409-a6b9a44da1b3 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:55.959510 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb4c9f7-c05a-4ca4-9771-1d79092cd71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:55.960365 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4c9f7-c05a-4ca4-9771-1d79092cd71e None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:55.960640 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4c9f7-c05a-4ca4-9771-1d79092cd71e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:55.960901 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4c9f7-c05a-4ca4-9771-1d79092cd71e None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:55.961549 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb4c9f7-c05a-4ca4-9771-1d79092cd71e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:55.963005 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb4c9f7-c05a-4ca4-9771-1d79092cd71e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:55.964754 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-a41be2a7-877b-4c8a-b409-a6b9a44da1b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:55.964754 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:55.964754 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:55.964754 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:55.964754 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:55.964754 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:55.964754 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:55.964754 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:55.964754 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:55.964754 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:55.964754 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:55.964754 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:55.964754 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:55.964754 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:55.964754 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:55.964754 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:55.965781 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:55.965781 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:55.965781 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:55.965781 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:55.965781 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:55.965781 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:55.965781 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:55.965781 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:55.965781 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:55.965781 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:55.965781 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:55.965781 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:55.965781 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:55.965781 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:55.965781 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:55.965781 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:55.965781 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:55.966715 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:55.966715 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:55.966715 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:55.966901 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: WARNING keystone.server.flask.common [None req-dfb4c9f7-c05a-4ca4-9771-1d79092cd71e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 31 05:06:55.971001 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 467/951] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:55.971831 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41be2a7-877b-4c8a-b409-a6b9a44da1b3 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:55.991670 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.sql.core [None req-dfb4c9f7-c05a-4ca4-9771-1d79092cd71e None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 05:06:55.991670 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 31 05:06:55.991670 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [parameters: {'id': '67bed140919a48168a99baa8dc64e964', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 31 05:06:55.991670 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81602) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 05:06:56.000000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-dfb4c9f7-c05a-4ca4-9771-1d79092cd71e None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 67bed140919a48168a99baa8dc64e964.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 67bed140919a48168a99baa8dc64e964. Aug 31 05:06:56.000000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:56.000000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 05:06:56.000000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 05:06:56.000000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 05:06:56.000000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 05:06:56.000000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 05:06:56.000000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 05:06:56.000000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 05:06:56.000000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application conn.query(q) Aug 31 05:06:56.000000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 05:06:56.000000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 05:06:56.000000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 05:06:56.000797 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application result.read() Aug 31 05:06:56.000797 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 05:06:56.000797 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 05:06:56.000797 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 05:06:56.000797 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 05:06:56.000797 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 05:06:56.000797 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 05:06:56.000797 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 05:06:56.000797 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 05:06:56.000797 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 05:06:56.000797 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:56.000797 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 05:06:56.000797 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:56.000797 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:56.000797 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 05:06:56.000797 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 05:06:56.000797 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 31 05:06:56.000797 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 31 05:06:56.001779 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 05:06:56.001779 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application next(self.gen) Aug 31 05:06:56.001779 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 05:06:56.001779 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application yield resource Aug 31 05:06:56.001779 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 05:06:56.001779 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application next(self.gen) Aug 31 05:06:56.001779 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 05:06:56.001779 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 05:06:56.001779 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 05:06:56.001779 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 05:06:56.001779 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 05:06:56.001779 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise self.value Aug 31 05:06:56.001779 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 05:06:56.001779 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 05:06:56.001779 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 05:06:56.001779 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application session.commit() Aug 31 05:06:56.001779 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 05:06:56.003194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 05:06:56.003194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 05:06:56.003194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 05:06:56.003194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 05:06:56.003194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self.session.flush() Aug 31 05:06:56.003194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 05:06:56.003194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 05:06:56.003194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 05:06:56.003194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 05:06:56.003194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 05:06:56.003194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application compat.raise_( Aug 31 05:06:56.003194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 05:06:56.003194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception Aug 31 05:06:56.003194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 05:06:56.003194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 05:06:56.003194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 05:06:56.003194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 05:06:56.004134 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 05:06:56.004134 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 05:06:56.004134 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 05:06:56.004134 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 05:06:56.004134 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 31 05:06:56.004134 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 31 05:06:56.004134 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 05:06:56.004134 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 05:06:56.004134 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 05:06:56.004134 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 05:06:56.004134 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 05:06:56.004134 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 05:06:56.004134 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 05:06:56.004134 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 05:06:56.004134 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 05:06:56.004929 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 05:06:56.004929 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 05:06:56.004929 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception Aug 31 05:06:56.004929 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 05:06:56.004929 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 05:06:56.004929 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 05:06:56.004929 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 05:06:56.004929 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 05:06:56.004929 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 05:06:56.004929 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 05:06:56.004929 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application conn.query(q) Aug 31 05:06:56.004929 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 05:06:56.004929 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 05:06:56.004929 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 05:06:56.004929 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application result.read() Aug 31 05:06:56.004929 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 05:06:56.004929 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 05:06:56.005788 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 05:06:56.005788 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 05:06:56.005788 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 05:06:56.005788 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 05:06:56.005788 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 05:06:56.005788 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 05:06:56.005788 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 05:06:56.005788 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 31 05:06:56.005788 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [parameters: {'id': '67bed140919a48168a99baa8dc64e964', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 31 05:06:56.005788 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 05:06:56.005788 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:56.005788 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 05:06:56.005788 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:56.005788 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:56.005788 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 31 05:06:56.008549 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 31 05:06:56.008549 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 05:06:56.008549 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 05:06:56.008549 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 31 05:06:56.008549 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:56.008549 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 05:06:56.008549 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:56.008549 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:56.008549 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:56.008549 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:56.008549 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:56.008549 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:56.008549 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:56.008549 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:56.008549 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:56.008549 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:56.008549 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:56.011185 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:56.011185 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 31 05:06:56.011185 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 31 05:06:56.011185 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:56.011185 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:56.011185 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 31 05:06:56.011185 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 31 05:06:56.011185 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 67bed140919a48168a99baa8dc64e964. Aug 31 05:06:56.011185 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:56.011185 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 485/952] 10.222.0.36 () {60 vars in 1140 bytes} [Wed Aug 31 05:06:55 2022] POST /identity/v3/projects => generated 262 bytes in 220 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:06:56.011185 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb4c9f7-c05a-4ca4-9771-1d79092cd71e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:56.021212 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:56.117954 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41be2a7-877b-4c8a-b409-a6b9a44da1b3 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:56.122243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb4c9f7-c05a-4ca4-9771-1d79092cd71e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:56.140184 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:56.164957 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b879806e-af7a-48dd-b859-c81c21f22be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:56.165622 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879806e-af7a-48dd-b859-c81c21f22be5 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:56.165902 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879806e-af7a-48dd-b859-c81c21f22be5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:56.166197 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879806e-af7a-48dd-b859-c81c21f22be5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:56.174302 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47360ec-fcc7-443e-a0ba-4e7e16ea8c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:56.174850 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47360ec-fcc7-443e-a0ba-4e7e16ea8c5d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:56.175123 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47360ec-fcc7-443e-a0ba-4e7e16ea8c5d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:56.175389 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47360ec-fcc7-443e-a0ba-4e7e16ea8c5d None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:56.175696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b879806e-af7a-48dd-b859-c81c21f22be5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:56.176039 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47360ec-fcc7-443e-a0ba-4e7e16ea8c5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:56.176337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47360ec-fcc7-443e-a0ba-4e7e16ea8c5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:56.177867 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47360ec-fcc7-443e-a0ba-4e7e16ea8c5d None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:56.178392 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b879806e-af7a-48dd-b859-c81c21f22be5 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:56.190587 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 468/953] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:56.191879 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d7e72-6fbf-40b4-9b41-b481f6d79226 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:56.192164 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d7e72-6fbf-40b4-9b41-b481f6d79226 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:56.192432 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d7e72-6fbf-40b4-9b41-b481f6d79226 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:56.193854 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 469/954] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:56.194416 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-b879806e-af7a-48dd-b859-c81c21f22be5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:56.194416 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:56.194416 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:56.194416 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:56.194416 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:56.194416 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:56.194416 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:56.194416 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:56.194416 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:56.194416 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:56.194416 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:56.194416 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:56.194416 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 05:06:56.194416 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 05:06:56.194416 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 05:06:56.195528 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 05:06:56.195528 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:56.195528 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:56.195528 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:56.195528 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:56.195528 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:56.195528 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:56.195528 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:56.195528 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:56.195528 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:56.195528 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:56.195528 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:56.195528 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:56.195528 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:56.195528 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:56.195528 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:56.195528 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:56.196496 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 05:06:56.196496 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 05:06:56.196496 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 05:06:56.196496 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 05:06:56.196496 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 05:06:56.196496 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 05:06:56.196496 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:56.196496 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:56.196960 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d7e72-6fbf-40b4-9b41-b481f6d79226 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:56.198526 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 486/955] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:06:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:56.200004 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921028ea-e7ee-420b-a990-343cf3558c75 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:56.200737 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921028ea-e7ee-420b-a990-343cf3558c75 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:56.201156 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921028ea-e7ee-420b-a990-343cf3558c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:56.221825 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-921028ea-e7ee-420b-a990-343cf3558c75 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:56.223355 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-921028ea-e7ee-420b-a990-343cf3558c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:56.284758 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:56.293635 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 487/956] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:06:56.294518 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921028ea-e7ee-420b-a990-343cf3558c75 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:56.316176 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d7e72-6fbf-40b4-9b41-b481f6d79226 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:56.363810 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2433be-4e64-405f-8685-57b6eb8c2c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:56.364833 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2433be-4e64-405f-8685-57b6eb8c2c5f None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:56.365905 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2433be-4e64-405f-8685-57b6eb8c2c5f None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:56.366433 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2433be-4e64-405f-8685-57b6eb8c2c5f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:56.376064 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2433be-4e64-405f-8685-57b6eb8c2c5f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:56.378383 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2433be-4e64-405f-8685-57b6eb8c2c5f None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:56.394395 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-6c2433be-4e64-405f-8685-57b6eb8c2c5f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 05:06:56.394395 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:56.394395 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:56.394395 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:56.394395 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:56.394395 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:56.394395 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:56.394395 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:56.394395 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:56.394395 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:56.394395 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:56.394395 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:56.394395 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 05:06:56.394395 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 05:06:56.394395 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 05:06:56.394395 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 05:06:56.395452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:56.395452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:56.395452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 05:06:56.395452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:56.395452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 05:06:56.395452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:56.395452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:56.395452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:56.395452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:56.395452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:56.395452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:56.395452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:56.395452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:56.395452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:56.395452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:56.395452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:56.395452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:56.395452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:56.396426 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:56.396426 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:56.396426 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 05:06:56.396426 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 05:06:56.396426 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 05:06:56.396426 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 05:06:56.396426 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 05:06:56.396426 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 05:06:56.396426 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 05:06:56.396426 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:56.399309 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 470/957] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Aug 31 05:06:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:56.404784 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2433be-4e64-405f-8685-57b6eb8c2c5f None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:56.412198 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921028ea-e7ee-420b-a990-343cf3558c75 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:56.446491 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:56.457975 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22d363d-9946-4442-ab8c-c4ca91469acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:56.459232 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22d363d-9946-4442-ab8c-c4ca91469acd None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:56.459839 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22d363d-9946-4442-ab8c-c4ca91469acd None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:56.460751 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22d363d-9946-4442-ab8c-c4ca91469acd None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:56.461890 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22d363d-9946-4442-ab8c-c4ca91469acd None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:56.462415 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22d363d-9946-4442-ab8c-c4ca91469acd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:56.464660 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22d363d-9946-4442-ab8c-c4ca91469acd None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:56.491235 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 488/958] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Aug 31 05:06:56 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:56.526344 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2433be-4e64-405f-8685-57b6eb8c2c5f None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:56.557862 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22d363d-9946-4442-ab8c-c4ca91469acd None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:56.567153 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99bba63-6f18-4ca4-a2a8-edf73d91cc73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:56.568072 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99bba63-6f18-4ca4-a2a8-edf73d91cc73 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:56.568510 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99bba63-6f18-4ca4-a2a8-edf73d91cc73 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:56.568922 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99bba63-6f18-4ca4-a2a8-edf73d91cc73 None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:56.569835 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99bba63-6f18-4ca4-a2a8-edf73d91cc73 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:56.570183 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99bba63-6f18-4ca4-a2a8-edf73d91cc73 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:56.571741 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99bba63-6f18-4ca4-a2a8-edf73d91cc73 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:56.591869 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 471/959] 10.222.0.36 () {56 vars in 1149 bytes} [Wed Aug 31 05:06:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:56.597428 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99bba63-6f18-4ca4-a2a8-edf73d91cc73 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:56.630492 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:56.692731 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22d363d-9946-4442-ab8c-c4ca91469acd None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:56.730759 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:56.731049 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328f3097-e7d7-4637-9e2b-e375e191a3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:56.731646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f3097-e7d7-4637-9e2b-e375e191a3cd None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:56.732593 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f3097-e7d7-4637-9e2b-e375e191a3cd None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:56.732593 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f3097-e7d7-4637-9e2b-e375e191a3cd None admin] PATH_INFO: `/v3/services/compute` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:56.732798 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328f3097-e7d7-4637-9e2b-e375e191a3cd None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:56.742092 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328f3097-e7d7-4637-9e2b-e375e191a3cd None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:56.742280 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99bba63-6f18-4ca4-a2a8-edf73d91cc73 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:56.743933 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-328f3097-e7d7-4637-9e2b-e375e191a3cd None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 05:06:56.743933 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:56.743933 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 05:06:56.743933 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 05:06:56.743933 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 05:06:56.743933 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 05:06:56.743933 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 05:06:56.743933 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:06:56.743933 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 05:06:56.743933 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:56.743933 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 05:06:56.743933 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:56.743933 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:56.743933 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:56.743933 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:56.743933 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:56.743933 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:56.745000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:56.745000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:56.745000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:56.745000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:56.745000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:56.745000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:56.745000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 05:06:56.745000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 05:06:56.745000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 05:06:56.745000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 05:06:56.745000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:56.745000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:56.745000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:56.745000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:56.745000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:56.745000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:56.745843 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:56.745843 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:56.745843 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:56.745843 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:56.745843 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:56.745843 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:56.745843 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:56.745843 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:56.745843 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:56.745843 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:56.745843 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 05:06:56.745843 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:06:56.745843 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 05:06:56.745843 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:56.745843 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 489/960] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:06:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:06:56.750969 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd75c60-1afd-46ef-abaa-9d36489df13e None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:56.750969 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd75c60-1afd-46ef-abaa-9d36489df13e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:56.750969 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd75c60-1afd-46ef-abaa-9d36489df13e None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:56.750969 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 490/961] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:56.753198 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd75c60-1afd-46ef-abaa-9d36489df13e None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:56.779369 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6897ab-4f7c-46f7-bdb1-e81963c00815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:56.779996 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6897ab-4f7c-46f7-bdb1-e81963c00815 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:56.780272 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6897ab-4f7c-46f7-bdb1-e81963c00815 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:56.780597 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6897ab-4f7c-46f7-bdb1-e81963c00815 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:56.855718 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd75c60-1afd-46ef-abaa-9d36489df13e None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:56.858160 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6897ab-4f7c-46f7-bdb1-e81963c00815 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:56.859824 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6897ab-4f7c-46f7-bdb1-e81963c00815 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:56.878508 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:56.895642 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e577a9-7cb7-4e48-8f36-bd7c8f702fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:56.896346 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e577a9-7cb7-4e48-8f36-bd7c8f702fac None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:56.896876 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e577a9-7cb7-4e48-8f36-bd7c8f702fac None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:56.897142 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e577a9-7cb7-4e48-8f36-bd7c8f702fac None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:56.898097 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e577a9-7cb7-4e48-8f36-bd7c8f702fac None admin] RBAC: Adding query filter params (name=compute) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:56.898399 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e577a9-7cb7-4e48-8f36-bd7c8f702fac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:56.899768 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e577a9-7cb7-4e48-8f36-bd7c8f702fac None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:56.904794 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 491/962] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Aug 31 05:06:56 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:56.905948 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206c3fd-0b92-4ef4-a6b9-61972776c784 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:56.906255 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206c3fd-0b92-4ef4-a6b9-61972776c784 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:56.906524 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206c3fd-0b92-4ef4-a6b9-61972776c784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:56.924908 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-9206c3fd-0b92-4ef4-a6b9-61972776c784 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:56.926109 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9206c3fd-0b92-4ef4-a6b9-61972776c784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:56.967355 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:56.975529 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 472/963] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:06:56 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 05:06:56.980246 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6897ab-4f7c-46f7-bdb1-e81963c00815 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:57.006051 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 492/964] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:06:57.006910 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206c3fd-0b92-4ef4-a6b9-61972776c784 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:57.095517 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6897ab-4f7c-46f7-bdb1-e81963c00815 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:57.122262 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206c3fd-0b92-4ef4-a6b9-61972776c784 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:57.137888 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b492019-c24e-4629-99e6-19d739590fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:57.138790 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b492019-c24e-4629-99e6-19d739590fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:57.139165 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b492019-c24e-4629-99e6-19d739590fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:57.139437 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:57.139653 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b492019-c24e-4629-99e6-19d739590fc5 None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:57.140407 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b492019-c24e-4629-99e6-19d739590fc5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:57.141065 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b492019-c24e-4629-99e6-19d739590fc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:57.142735 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b492019-c24e-4629-99e6-19d739590fc5 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:57.155098 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 473/965] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Aug 31 05:06:56 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:57.165185 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ca514-634b-4d84-841f-766f6a7b2bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:57.165841 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ca514-634b-4d84-841f-766f6a7b2bde None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:57.166157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ca514-634b-4d84-841f-766f6a7b2bde None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:57.166425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ca514-634b-4d84-841f-766f6a7b2bde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:57.178982 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28ca514-634b-4d84-841f-766f6a7b2bde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:57.180467 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28ca514-634b-4d84-841f-766f6a7b2bde None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:57.197425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-e28ca514-634b-4d84-841f-766f6a7b2bde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:57.197425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:57.197425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:57.197425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:57.197425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:57.197425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:57.197425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:57.197425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:57.197425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:57.197425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:57.197425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:57.197425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:57.197425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:06:57.197425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:06:57.197425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:06:57.197425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:06:57.198486 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:57.198486 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:57.198486 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:57.198486 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:57.198486 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:57.198486 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:57.198486 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:57.198486 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:57.198486 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:57.198486 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:57.198486 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:57.198486 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:57.198486 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:57.198486 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:57.198486 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:57.198486 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:57.198486 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:06:57.199401 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:06:57.199401 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:06:57.199401 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:57.199401 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 493/966] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:06:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:57.204586 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b492019-c24e-4629-99e6-19d739590fc5 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:57.267633 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:57.276161 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ca514-634b-4d84-841f-766f6a7b2bde None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:57.317060 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b492019-c24e-4629-99e6-19d739590fc5 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:57.364562 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c06dd02-c356-4dfb-a2cd-225601a258cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:57.365518 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c06dd02-c356-4dfb-a2cd-225601a258cd None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:57.365984 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c06dd02-c356-4dfb-a2cd-225601a258cd None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:57.366478 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c06dd02-c356-4dfb-a2cd-225601a258cd None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:57.367367 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c06dd02-c356-4dfb-a2cd-225601a258cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:57.368166 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c06dd02-c356-4dfb-a2cd-225601a258cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:57.369744 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c06dd02-c356-4dfb-a2cd-225601a258cd None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:57.383250 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 474/967] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:06:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:57.389316 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c06dd02-c356-4dfb-a2cd-225601a258cd None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:57.407007 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ca514-634b-4d84-841f-766f6a7b2bde None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:57.450316 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f4fc6-3a1a-4c18-9a54-3a00341592a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:57.451306 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f4fc6-3a1a-4c18-9a54-3a00341592a4 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:57.452146 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f4fc6-3a1a-4c18-9a54-3a00341592a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:57.452629 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f4fc6-3a1a-4c18-9a54-3a00341592a4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:57.453641 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6f4fc6-3a1a-4c18-9a54-3a00341592a4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:57.455826 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6f4fc6-3a1a-4c18-9a54-3a00341592a4 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:57.467036 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-2e6f4fc6-3a1a-4c18-9a54-3a00341592a4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 05:06:57.467036 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:57.467036 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 05:06:57.467036 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 05:06:57.467036 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 05:06:57.467036 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 05:06:57.467036 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 05:06:57.467036 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:06:57.467036 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 05:06:57.467036 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:57.467036 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 05:06:57.467036 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:57.467036 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:57.467036 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:57.467036 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:57.467036 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:57.467036 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:57.468274 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:57.468274 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:57.468274 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:57.468274 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:57.468274 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:57.468274 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:57.468274 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 05:06:57.468274 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 05:06:57.468274 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 05:06:57.468274 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 05:06:57.468274 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:57.468274 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:57.468274 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:57.468274 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:57.468274 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:57.468274 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:57.469308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:57.469308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:57.469308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:57.469308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:57.469308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:57.469308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:57.469308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:57.469308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:57.469308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:57.469308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:57.469308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 05:06:57.469308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:06:57.469308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 05:06:57.469308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:57.471411 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 494/968] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Aug 31 05:06:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:06:57.478126 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f4fc6-3a1a-4c18-9a54-3a00341592a4 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:57.511219 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:57.519397 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c06dd02-c356-4dfb-a2cd-225601a258cd None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:57.542329 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:57.566424 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2872bd22-9c4c-4498-9e32-0841fb316e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:57.567883 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2872bd22-9c4c-4498-9e32-0841fb316e6c None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:57.568328 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2872bd22-9c4c-4498-9e32-0841fb316e6c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:57.569170 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2872bd22-9c4c-4498-9e32-0841fb316e6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:57.578997 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2872bd22-9c4c-4498-9e32-0841fb316e6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:57.581219 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2872bd22-9c4c-4498-9e32-0841fb316e6c None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:57.587803 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f4fc6-3a1a-4c18-9a54-3a00341592a4 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:57.596315 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-2872bd22-9c4c-4498-9e32-0841fb316e6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:57.596315 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:57.596315 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:57.596315 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:57.596315 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:57.596315 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:57.596315 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:57.596315 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:57.596315 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:57.596315 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:57.596315 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:57.596315 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:57.596315 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 05:06:57.596315 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 05:06:57.596315 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 05:06:57.597449 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 05:06:57.597449 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:57.597449 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:57.597449 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:57.597449 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:57.597449 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:57.597449 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:57.597449 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:57.597449 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:57.597449 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:57.597449 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:57.597449 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:57.597449 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:57.597449 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:57.597449 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:57.597449 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:57.597449 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:57.598574 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 05:06:57.598574 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 05:06:57.598574 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 05:06:57.598574 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 05:06:57.598574 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 05:06:57.598574 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 05:06:57.598574 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:06:57.598574 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:57.600637 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 475/969] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:06:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:57.606038 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2872bd22-9c4c-4498-9e32-0841fb316e6c None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:57.629802 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3601f5ee-26f2-4485-bf3c-7b12f1d00383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:57.630740 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3601f5ee-26f2-4485-bf3c-7b12f1d00383 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:57.631732 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3601f5ee-26f2-4485-bf3c-7b12f1d00383 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:57.632172 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3601f5ee-26f2-4485-bf3c-7b12f1d00383 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:57.633076 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3601f5ee-26f2-4485-bf3c-7b12f1d00383 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:57.633559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3601f5ee-26f2-4485-bf3c-7b12f1d00383 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:57.635442 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3601f5ee-26f2-4485-bf3c-7b12f1d00383 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:57.644040 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 495/970] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Aug 31 05:06:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:06:57.649341 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3601f5ee-26f2-4485-bf3c-7b12f1d00383 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:57.738320 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2872bd22-9c4c-4498-9e32-0841fb316e6c None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:57.759882 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:57.770898 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3601f5ee-26f2-4485-bf3c-7b12f1d00383 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:57.775259 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b64fa23-02b5-4430-9833-c32a527eed0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:57.775860 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b64fa23-02b5-4430-9833-c32a527eed0d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:57.776136 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b64fa23-02b5-4430-9833-c32a527eed0d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:57.776398 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b64fa23-02b5-4430-9833-c32a527eed0d None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:57.777138 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b64fa23-02b5-4430-9833-c32a527eed0d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:57.777440 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b64fa23-02b5-4430-9833-c32a527eed0d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:57.778954 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b64fa23-02b5-4430-9833-c32a527eed0d None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:57.791808 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 476/971] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 05:06:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:57.797350 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b64fa23-02b5-4430-9833-c32a527eed0d None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:57.812663 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b51ad2-d79f-437e-83f2-2743d3ed3e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:57.813941 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b51ad2-d79f-437e-83f2-2743d3ed3e87 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:57.814636 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b51ad2-d79f-437e-83f2-2743d3ed3e87 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:57.815054 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b51ad2-d79f-437e-83f2-2743d3ed3e87 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:57.816004 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b51ad2-d79f-437e-83f2-2743d3ed3e87 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:57.816453 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b51ad2-d79f-437e-83f2-2743d3ed3e87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:57.818252 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b51ad2-d79f-437e-83f2-2743d3ed3e87 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:57.829611 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 496/972] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Aug 31 05:06:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:57.830707 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b51ad2-d79f-437e-83f2-2743d3ed3e87 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:57.840590 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:57.903354 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b64fa23-02b5-4430-9833-c32a527eed0d None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:57.935609 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b51ad2-d79f-437e-83f2-2743d3ed3e87 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:57.939929 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9554f838-2e94-4ffb-94d1-02475d135e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:57.940653 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554f838-2e94-4ffb-94d1-02475d135e46 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:57.941025 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554f838-2e94-4ffb-94d1-02475d135e46 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:57.941392 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554f838-2e94-4ffb-94d1-02475d135e46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:57.942247 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9554f838-2e94-4ffb-94d1-02475d135e46 None admin] RBAC: Adding query filter params (role.id=f7c0c9eac46c42179e586dc4c1726356, scope.project.id=e01ce1b1282e44a39d659b87f120995e, user.id=09bf6db88ce8494a937211a814c9ff84) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:57.942651 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9554f838-2e94-4ffb-94d1-02475d135e46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:57.944244 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9554f838-2e94-4ffb-94d1-02475d135e46 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:57.958482 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 477/973] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:06:57 2022] GET /identity/v3/role_assignments?user.id=09bf6db88ce8494a937211a814c9ff84&scope.project.id=e01ce1b1282e44a39d659b87f120995e&role.id=f7c0c9eac46c42179e586dc4c1726356 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:57.973973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d094c11-314e-480e-8686-f63091e236ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:57.974892 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d094c11-314e-480e-8686-f63091e236ca None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:57.975339 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d094c11-314e-480e-8686-f63091e236ca None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:57.975749 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d094c11-314e-480e-8686-f63091e236ca None admin] PATH_INFO: `/v3/endpoints` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:57.976601 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d094c11-314e-480e-8686-f63091e236ca None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:57.978287 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d094c11-314e-480e-8686-f63091e236ca None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:58.004310 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 497/974] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:06:57 2022] POST /identity/v3/endpoints => generated 349 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:06:58.511484 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78df8ca-c4f2-4c97-9da1-5b83aa1f4286 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:58.512042 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78df8ca-c4f2-4c97-9da1-5b83aa1f4286 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:58.512042 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78df8ca-c4f2-4c97-9da1-5b83aa1f4286 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:58.513685 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 478/975] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:58.518917 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9de80-0302-4c9b-a46d-eb7e40aaeba5 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:58.519201 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9de80-0302-4c9b-a46d-eb7e40aaeba5 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:58.519471 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9de80-0302-4c9b-a46d-eb7e40aaeba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:58.537755 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-bfa9de80-0302-4c9b-a46d-eb7e40aaeba5 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:58.538951 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-bfa9de80-0302-4c9b-a46d-eb7e40aaeba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:58.548669 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:58.618971 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 498/976] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:06:58.897091 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78df8ca-c4f2-4c97-9da1-5b83aa1f4286 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:59.001518 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78df8ca-c4f2-4c97-9da1-5b83aa1f4286 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:59.003245 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:59.037232 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5007baf-92b6-4246-99f9-4962fd4ec7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:59.037837 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5007baf-92b6-4246-99f9-4962fd4ec7d2 None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:59.038211 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5007baf-92b6-4246-99f9-4962fd4ec7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:59.038476 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5007baf-92b6-4246-99f9-4962fd4ec7d2 None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:59.039115 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5007baf-92b6-4246-99f9-4962fd4ec7d2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:59.040561 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5007baf-92b6-4246-99f9-4962fd4ec7d2 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:59.043119 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: WARNING keystone.server.flask.common [None req-b5007baf-92b6-4246-99f9-4962fd4ec7d2 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 31 05:06:59.057041 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.sql.core [None req-b5007baf-92b6-4246-99f9-4962fd4ec7d2 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 05:06:59.057041 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 31 05:06:59.057041 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [parameters: {'user_id': '878952954fa24618b7c3b9de12e808eb', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 31 05:06:59.057041 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81601) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 05:06:59.073863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-b5007baf-92b6-4246-99f9-4962fd4ec7d2 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 31 05:06:59.073863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:59.073863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 05:06:59.073863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 05:06:59.073863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 05:06:59.073863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 05:06:59.073863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 05:06:59.073863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 05:06:59.073863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 05:06:59.073863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application conn.query(q) Aug 31 05:06:59.073863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 05:06:59.073863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 05:06:59.073863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 05:06:59.073863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application result.read() Aug 31 05:06:59.073863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 05:06:59.074885 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 05:06:59.074885 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 05:06:59.074885 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 05:06:59.074885 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 05:06:59.074885 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 05:06:59.074885 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 05:06:59.074885 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 05:06:59.074885 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 05:06:59.074885 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:59.074885 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 05:06:59.074885 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:59.074885 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:59.074885 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 05:06:59.074885 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 05:06:59.074885 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 31 05:06:59.074885 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 31 05:06:59.074885 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 05:06:59.074885 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application next(self.gen) Aug 31 05:06:59.075862 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 05:06:59.075862 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application yield resource Aug 31 05:06:59.075862 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 05:06:59.075862 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application next(self.gen) Aug 31 05:06:59.075862 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 05:06:59.075862 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 05:06:59.075862 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 05:06:59.075862 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 05:06:59.075862 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 05:06:59.075862 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise self.value Aug 31 05:06:59.075862 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 05:06:59.075862 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 05:06:59.075862 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 05:06:59.075862 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application session.commit() Aug 31 05:06:59.075862 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 05:06:59.075862 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 05:06:59.075862 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 05:06:59.076765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 05:06:59.076765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 05:06:59.076765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self.session.flush() Aug 31 05:06:59.076765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 05:06:59.076765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 05:06:59.076765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 05:06:59.076765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 05:06:59.076765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 05:06:59.076765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application compat.raise_( Aug 31 05:06:59.076765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 05:06:59.076765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception Aug 31 05:06:59.076765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 05:06:59.076765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 05:06:59.076765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 05:06:59.076765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 05:06:59.076765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 05:06:59.076765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 05:06:59.077689 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 05:06:59.077689 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 05:06:59.077689 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 31 05:06:59.077689 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 31 05:06:59.077689 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 05:06:59.077689 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 05:06:59.077689 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 05:06:59.077689 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 05:06:59.077689 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 05:06:59.077689 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 05:06:59.077689 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 05:06:59.077689 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 05:06:59.077689 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 05:06:59.077689 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 05:06:59.077689 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 05:06:59.077689 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception Aug 31 05:06:59.082953 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 05:06:59.082953 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 05:06:59.082953 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 05:06:59.082953 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 05:06:59.082953 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 05:06:59.082953 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 05:06:59.082953 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 05:06:59.082953 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application conn.query(q) Aug 31 05:06:59.082953 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 05:06:59.082953 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 05:06:59.082953 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 05:06:59.082953 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application result.read() Aug 31 05:06:59.082953 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 05:06:59.082953 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 05:06:59.082953 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 05:06:59.082953 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 05:06:59.084877 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 05:06:59.084877 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 05:06:59.084877 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 05:06:59.084877 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 05:06:59.084877 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 05:06:59.084877 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 31 05:06:59.084877 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [parameters: {'user_id': '878952954fa24618b7c3b9de12e808eb', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 31 05:06:59.084877 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 05:06:59.084877 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:59.084877 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 05:06:59.084877 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:59.084877 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:59.084877 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:59.084877 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:59.084877 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:59.086749 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:59.086749 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:59.086749 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:59.086749 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:59.086749 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:59.086749 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:59.086749 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:59.086749 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 31 05:06:59.086749 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 31 05:06:59.086749 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:59.086749 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:59.086749 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 05:06:59.086749 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:59.086749 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 05:06:59.086749 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:59.086749 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 31 05:06:59.086749 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 31 05:06:59.088881 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 31 05:06:59.088881 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 31 05:06:59.088881 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 05:06:59.088881 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 05:06:59.088881 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 31 05:06:59.088881 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:06:59.088881 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 479/977] 10.222.0.36 () {60 vars in 1131 bytes} [Wed Aug 31 05:06:58 2022] POST /identity/v3/users => generated 160 bytes in 179 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:59.094089 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa9de80-0302-4c9b-a46d-eb7e40aaeba5 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:59.098312 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac520d-2fe2-4bb3-855f-d25871db449b None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:59.098583 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac520d-2fe2-4bb3-855f-d25871db449b None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:59.098852 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac520d-2fe2-4bb3-855f-d25871db449b None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:59.103410 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 480/978] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 05:06:59.109154 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b683d6a-78d2-44b8-8c72-f0b6ee6de515 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:59.109469 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b683d6a-78d2-44b8-8c72-f0b6ee6de515 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:59.109713 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b683d6a-78d2-44b8-8c72-f0b6ee6de515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:59.131163 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-7b683d6a-78d2-44b8-8c72-f0b6ee6de515 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:59.132885 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-7b683d6a-78d2-44b8-8c72-f0b6ee6de515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:59.210169 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 481/979] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:06:59.226592 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa9de80-0302-4c9b-a46d-eb7e40aaeba5 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:59.261613 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a056848-f27f-4440-b57c-5ba5ed8c97eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:59.262611 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a056848-f27f-4440-b57c-5ba5ed8c97eb None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:59.263105 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a056848-f27f-4440-b57c-5ba5ed8c97eb None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:59.263572 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a056848-f27f-4440-b57c-5ba5ed8c97eb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:59.270885 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a056848-f27f-4440-b57c-5ba5ed8c97eb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:59.272638 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a056848-f27f-4440-b57c-5ba5ed8c97eb None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:59.275407 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:59.286397 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-4a056848-f27f-4440-b57c-5ba5ed8c97eb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 05:06:59.286397 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:06:59.286397 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:06:59.286397 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:06:59.286397 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:06:59.286397 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:06:59.286397 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:06:59.286397 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:06:59.286397 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:06:59.286397 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:06:59.286397 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:06:59.286397 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:06:59.286397 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 05:06:59.286397 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 05:06:59.286397 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 05:06:59.286397 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 05:06:59.287367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:06:59.287367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:06:59.287367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 05:06:59.287367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:59.287367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 05:06:59.287367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:06:59.287367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:06:59.287367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:06:59.287367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:06:59.287367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:06:59.287367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:06:59.287367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:06:59.287367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:06:59.287367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:06:59.287367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:06:59.287367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:06:59.287367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:06:59.287367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:06:59.288330 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:06:59.288330 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:06:59.288330 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 05:06:59.288330 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 05:06:59.288330 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 05:06:59.288330 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 05:06:59.288330 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 05:06:59.288330 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 05:06:59.288330 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 05:06:59.288330 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:06:59.290934 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 499/980] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:06:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:06:59.296900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b683d6a-78d2-44b8-8c72-f0b6ee6de515 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:59.393941 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:06:59.402228 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b683d6a-78d2-44b8-8c72-f0b6ee6de515 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:59.436791 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9830c4-206b-4676-b8a7-7cf7128c9cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:59.437748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9830c4-206b-4676-b8a7-7cf7128c9cfd None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:59.438264 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9830c4-206b-4676-b8a7-7cf7128c9cfd None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:59.438722 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9830c4-206b-4676-b8a7-7cf7128c9cfd None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:59.439621 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9830c4-206b-4676-b8a7-7cf7128c9cfd None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:06:59.440122 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9830c4-206b-4676-b8a7-7cf7128c9cfd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:59.441879 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9830c4-206b-4676-b8a7-7cf7128c9cfd None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:59.476795 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a056848-f27f-4440-b57c-5ba5ed8c97eb None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:06:59.482333 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 482/981] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Aug 31 05:06:59 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:06:59.589095 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a056848-f27f-4440-b57c-5ba5ed8c97eb None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:06:59.629707 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49048e2-b1f2-4730-9e67-076695a5e37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:06:59.630714 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49048e2-b1f2-4730-9e67-076695a5e37c None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:59.631187 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49048e2-b1f2-4730-9e67-076695a5e37c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:59.631673 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49048e2-b1f2-4730-9e67-076695a5e37c None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:59.632554 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49048e2-b1f2-4730-9e67-076695a5e37c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:06:59.634200 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49048e2-b1f2-4730-9e67-076695a5e37c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:06:59.662670 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 500/982] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Aug 31 05:06:59 2022] POST /identity/v3/services => generated 242 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:06:59.950292 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8612d8c4-1af4-45e2-a3b0-9aa62f98d16e None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:59.950865 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8612d8c4-1af4-45e2-a3b0-9aa62f98d16e None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:59.950865 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8612d8c4-1af4-45e2-a3b0-9aa62f98d16e None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:59.952320 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 483/983] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:59.957363 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7183aa5a-ecbc-4025-a718-8a0b6d297118 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:59.957655 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7183aa5a-ecbc-4025-a718-8a0b6d297118 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:59.957910 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7183aa5a-ecbc-4025-a718-8a0b6d297118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:59.976708 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-7183aa5a-ecbc-4025-a718-8a0b6d297118 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:06:59.977576 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc6c4d2-977b-41cb-8d87-190d3a7fb468 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:59.977863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc6c4d2-977b-41cb-8d87-190d3a7fb468 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:59.977980 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7183aa5a-ecbc-4025-a718-8a0b6d297118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:06:59.978421 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc6c4d2-977b-41cb-8d87-190d3a7fb468 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:06:59.980283 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 484/984] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:06:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:06:59.985137 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bbdc36-f668-4038-8836-4895ec1cb625 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:06:59.985404 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bbdc36-f668-4038-8836-4895ec1cb625 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:06:59.985694 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bbdc36-f668-4038-8836-4895ec1cb625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:00.004844 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-95bbdc36-f668-4038-8836-4895ec1cb625 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:07:00.005971 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-95bbdc36-f668-4038-8836-4895ec1cb625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:07:00.036864 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:00.056632 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 501/985] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:59 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:07:00.082224 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 485/986] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:06:59 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 05:07:00.326683 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7183aa5a-ecbc-4025-a718-8a0b6d297118 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:00.347612 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bbdc36-f668-4038-8836-4895ec1cb625 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:00.429605 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7183aa5a-ecbc-4025-a718-8a0b6d297118 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:00.429795 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:00.450106 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bbdc36-f668-4038-8836-4895ec1cb625 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:00.464341 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d532c5-9603-4591-a73e-536eead707b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:00.464974 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d532c5-9603-4591-a73e-536eead707b2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:00.465310 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d532c5-9603-4591-a73e-536eead707b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:00.465579 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d532c5-9603-4591-a73e-536eead707b2 None admin] PATH_INFO: `/v3/services/image` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:00.466189 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d532c5-9603-4591-a73e-536eead707b2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:00.467564 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d532c5-9603-4591-a73e-536eead707b2 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:00.476856 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-10d532c5-9603-4591-a73e-536eead707b2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 05:07:00.476856 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:07:00.476856 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 05:07:00.476856 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 05:07:00.476856 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 05:07:00.476856 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 05:07:00.476856 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 05:07:00.476856 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:07:00.476856 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 05:07:00.476856 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:07:00.476856 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 05:07:00.476856 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:07:00.476856 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:07:00.476856 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:07:00.476856 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:07:00.476856 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:07:00.476856 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:07:00.477855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:07:00.477855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:07:00.477855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:07:00.477855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:07:00.477855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:07:00.477855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:07:00.477855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 05:07:00.477855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 05:07:00.477855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 05:07:00.477855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 05:07:00.477855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:07:00.477855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:07:00.477855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:07:00.477855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:07:00.477855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:07:00.477855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:07:00.478814 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:07:00.478814 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:07:00.478814 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:07:00.478814 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:07:00.478814 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:07:00.478814 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:07:00.478814 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:07:00.478814 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:07:00.478814 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:07:00.478814 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:07:00.478814 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 05:07:00.478814 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:07:00.478814 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 05:07:00.478814 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:07:00.478814 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 502/987] 10.222.0.36 () {56 vars in 1105 bytes} [Wed Aug 31 05:07:00 2022] GET /identity/v3/services/image => generated 86 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:07:00.483751 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d532c5-9603-4591-a73e-536eead707b2 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:00.489194 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb9b690-4441-4266-98d0-b5e2eeb85f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:00.489828 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9b690-4441-4266-98d0-b5e2eeb85f84 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:00.490147 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9b690-4441-4266-98d0-b5e2eeb85f84 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:00.490393 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9b690-4441-4266-98d0-b5e2eeb85f84 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:00.491000 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb9b690-4441-4266-98d0-b5e2eeb85f84 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:00.493096 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb9b690-4441-4266-98d0-b5e2eeb85f84 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:00.502918 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-bbb9b690-4441-4266-98d0-b5e2eeb85f84 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 05:07:00.502918 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:07:00.502918 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 05:07:00.502918 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 05:07:00.502918 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 05:07:00.502918 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 05:07:00.502918 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 05:07:00.502918 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:07:00.502918 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 05:07:00.502918 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:07:00.502918 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 05:07:00.502918 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:07:00.502918 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:07:00.502918 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:07:00.502918 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:07:00.502918 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:07:00.502918 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:07:00.503944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:07:00.503944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:07:00.503944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:07:00.503944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:07:00.503944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:07:00.503944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:07:00.503944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 05:07:00.503944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 05:07:00.503944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 05:07:00.503944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 05:07:00.503944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:07:00.503944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:07:00.503944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:07:00.503944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:07:00.503944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:07:00.503944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:07:00.504793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:07:00.504793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:07:00.504793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:07:00.504793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:07:00.504793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:07:00.504793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:07:00.504793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:07:00.504793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:07:00.504793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:07:00.504793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:07:00.504793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 05:07:00.504793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:07:00.504793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 05:07:00.504793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:07:00.508975 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 486/988] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Aug 31 05:07:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:07:00.509826 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb9b690-4441-4266-98d0-b5e2eeb85f84 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:00.537104 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:00.590218 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d532c5-9603-4591-a73e-536eead707b2 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:00.616763 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb9b690-4441-4266-98d0-b5e2eeb85f84 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:00.624388 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e934c99-06b6-47cd-a380-36af0df45d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:00.625001 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e934c99-06b6-47cd-a380-36af0df45d9c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:00.625275 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e934c99-06b6-47cd-a380-36af0df45d9c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:00.625538 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e934c99-06b6-47cd-a380-36af0df45d9c None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:00.626196 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e934c99-06b6-47cd-a380-36af0df45d9c None admin] RBAC: Adding query filter params (name=image) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:00.626492 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e934c99-06b6-47cd-a380-36af0df45d9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:00.627852 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e934c99-06b6-47cd-a380-36af0df45d9c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:00.632949 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 503/989] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:07:00 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:07:00.639152 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e934c99-06b6-47cd-a380-36af0df45d9c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:00.666586 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385d023d-2b68-4540-b4c9-1e07ce5f8123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:00.668131 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d023d-2b68-4540-b4c9-1e07ce5f8123 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:00.668638 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d023d-2b68-4540-b4c9-1e07ce5f8123 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:00.669072 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d023d-2b68-4540-b4c9-1e07ce5f8123 None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:00.670176 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385d023d-2b68-4540-b4c9-1e07ce5f8123 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:00.670654 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385d023d-2b68-4540-b4c9-1e07ce5f8123 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:00.673061 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385d023d-2b68-4540-b4c9-1e07ce5f8123 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:00.682886 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 487/990] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Aug 31 05:07:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:07:00.686202 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385d023d-2b68-4540-b4c9-1e07ce5f8123 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:00.699187 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:00.753974 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e934c99-06b6-47cd-a380-36af0df45d9c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:00.770148 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:00.792405 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3128f18-2e84-4384-929d-433c71905e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:00.792692 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385d023d-2b68-4540-b4c9-1e07ce5f8123 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:00.793154 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3128f18-2e84-4384-929d-433c71905e43 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:00.793577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3128f18-2e84-4384-929d-433c71905e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:00.793901 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3128f18-2e84-4384-929d-433c71905e43 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:00.794530 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3128f18-2e84-4384-929d-433c71905e43 None admin] RBAC: Adding query filter params (type=image) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:00.794842 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3128f18-2e84-4384-929d-433c71905e43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:00.796533 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3128f18-2e84-4384-929d-433c71905e43 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:00.801391 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 504/991] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:07:00 2022] GET /identity/v3/services?type=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:07:00.829636 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf160df7-a03d-4916-b6e4-91955acd8f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:00.830298 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf160df7-a03d-4916-b6e4-91955acd8f37 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:00.830573 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf160df7-a03d-4916-b6e4-91955acd8f37 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:00.830844 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf160df7-a03d-4916-b6e4-91955acd8f37 None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:00.831482 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf160df7-a03d-4916-b6e4-91955acd8f37 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:00.831783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf160df7-a03d-4916-b6e4-91955acd8f37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:00.833199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf160df7-a03d-4916-b6e4-91955acd8f37 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:00.839283 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 488/992] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Aug 31 05:07:00 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:00.845128 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3128f18-2e84-4384-929d-433c71905e43 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:00.959435 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3128f18-2e84-4384-929d-433c71905e43 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:00.997079 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb974b-527b-4223-a4c3-31f138658fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:00.997693 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb974b-527b-4223-a4c3-31f138658fe2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:00.997969 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb974b-527b-4223-a4c3-31f138658fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:00.998269 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb974b-527b-4223-a4c3-31f138658fe2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:00.998981 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eb974b-527b-4223-a4c3-31f138658fe2 None admin] RBAC: Adding query filter params (interface=admin, service_id=64bf473e5bec4e8090afac8ce11b3add, region_id=RegionOne) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:00.999287 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eb974b-527b-4223-a4c3-31f138658fe2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:01.000655 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eb974b-527b-4223-a4c3-31f138658fe2 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:01.010066 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 505/993] 10.222.0.36 () {56 vars in 1249 bytes} [Wed Aug 31 05:07:00 2022] GET /identity/v3/endpoints?service_id=64bf473e5bec4e8090afac8ce11b3add&interface=admin®ion_id=RegionOne => generated 193 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:01.011747 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf160df7-a03d-4916-b6e4-91955acd8f37 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:01.089667 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:01.116979 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf160df7-a03d-4916-b6e4-91955acd8f37 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:01.152084 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cf472c-d8f4-4d2b-87b7-debeeae0a73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:01.152780 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf472c-d8f4-4d2b-87b7-debeeae0a73e None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:01.153054 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf472c-d8f4-4d2b-87b7-debeeae0a73e None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:01.153318 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf472c-d8f4-4d2b-87b7-debeeae0a73e None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:01.153916 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cf472c-d8f4-4d2b-87b7-debeeae0a73e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:01.155316 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cf472c-d8f4-4d2b-87b7-debeeae0a73e None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:01.162040 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 489/994] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Aug 31 05:07:01 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:07:01.570925 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c91a99-8921-472e-800e-91f37da40cf6 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:01.571467 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c91a99-8921-472e-800e-91f37da40cf6 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:01.571467 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c91a99-8921-472e-800e-91f37da40cf6 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:01.572972 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 506/995] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:07:01.579195 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ad669-ab0a-41bc-8db9-8059b37f9640 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:01.579660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ad669-ab0a-41bc-8db9-8059b37f9640 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:01.580160 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ad669-ab0a-41bc-8db9-8059b37f9640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:01.601666 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-536ad669-ab0a-41bc-8db9-8059b37f9640 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:07:01.602885 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-536ad669-ab0a-41bc-8db9-8059b37f9640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:07:01.680636 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 490/996] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:07:01 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:07:01.946252 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c91a99-8921-472e-800e-91f37da40cf6 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:01.992443 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:02.050174 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c91a99-8921-472e-800e-91f37da40cf6 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:02.082617 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca17ad-2b33-4e5e-9a5a-55cba424f73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:02.083237 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca17ad-2b33-4e5e-9a5a-55cba424f73b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:02.083510 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca17ad-2b33-4e5e-9a5a-55cba424f73b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:02.083772 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca17ad-2b33-4e5e-9a5a-55cba424f73b None admin] PATH_INFO: `/v3/services/compute` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:02.084344 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ca17ad-2b33-4e5e-9a5a-55cba424f73b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:02.085712 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ca17ad-2b33-4e5e-9a5a-55cba424f73b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:02.094418 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-69ca17ad-2b33-4e5e-9a5a-55cba424f73b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 05:07:02.094418 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:07:02.094418 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 05:07:02.094418 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 05:07:02.094418 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 05:07:02.094418 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 05:07:02.094418 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 05:07:02.094418 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:07:02.094418 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 05:07:02.094418 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:07:02.094418 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 05:07:02.094418 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:07:02.094418 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:07:02.094418 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:07:02.094418 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:07:02.094418 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:07:02.094418 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:07:02.095425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:07:02.095425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:07:02.095425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:07:02.095425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:07:02.095425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:07:02.095425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:07:02.095425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 05:07:02.095425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 05:07:02.095425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 05:07:02.095425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 05:07:02.095425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:07:02.095425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:07:02.095425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:07:02.095425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:07:02.095425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:07:02.095425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:07:02.096311 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:07:02.096311 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:07:02.096311 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:07:02.096311 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:07:02.096311 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:07:02.096311 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:07:02.096311 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:07:02.096311 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:07:02.096311 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:07:02.096311 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:07:02.096311 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 05:07:02.096311 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:07:02.096311 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 05:07:02.096311 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:07:02.098173 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 507/997] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:07:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:07:02.103515 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ad669-ab0a-41bc-8db9-8059b37f9640 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:02.170745 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:02.206230 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ad669-ab0a-41bc-8db9-8059b37f9640 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:02.244602 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e935b-3272-4da0-9e1b-86b015ed4906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:02.245578 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e935b-3272-4da0-9e1b-86b015ed4906 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:02.246087 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e935b-3272-4da0-9e1b-86b015ed4906 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:02.246650 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e935b-3272-4da0-9e1b-86b015ed4906 None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:02.247495 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3e935b-3272-4da0-9e1b-86b015ed4906 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:02.248012 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3e935b-3272-4da0-9e1b-86b015ed4906 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:02.249753 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3e935b-3272-4da0-9e1b-86b015ed4906 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:02.254893 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 491/998] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Aug 31 05:07:02 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:02.260676 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca17ad-2b33-4e5e-9a5a-55cba424f73b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:02.364435 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca17ad-2b33-4e5e-9a5a-55cba424f73b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:02.372335 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:02.401277 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9c7c98-6a2c-4e2d-afa0-1fef711f2219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:02.401890 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c7c98-6a2c-4e2d-afa0-1fef711f2219 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:02.402179 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c7c98-6a2c-4e2d-afa0-1fef711f2219 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:02.402472 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c7c98-6a2c-4e2d-afa0-1fef711f2219 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:02.403125 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9c7c98-6a2c-4e2d-afa0-1fef711f2219 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:02.403425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9c7c98-6a2c-4e2d-afa0-1fef711f2219 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:02.404782 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9c7c98-6a2c-4e2d-afa0-1fef711f2219 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:02.414381 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 508/999] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Aug 31 05:07:02 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:02.415585 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e935b-3272-4da0-9e1b-86b015ed4906 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:02.518138 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e935b-3272-4da0-9e1b-86b015ed4906 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:02.549919 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:02.556109 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b7ad5-85f5-4a6b-b12f-0cfb80a42f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:02.556719 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b7ad5-85f5-4a6b-b12f-0cfb80a42f24 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:02.556989 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b7ad5-85f5-4a6b-b12f-0cfb80a42f24 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:02.557252 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b7ad5-85f5-4a6b-b12f-0cfb80a42f24 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:02.557953 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98b7ad5-85f5-4a6b-b12f-0cfb80a42f24 None admin] RBAC: Adding query filter params (interface=public, service_id=fb30a2e41b7847a7950e667b5a35e778, region_id=RegionOne) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:02.558281 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98b7ad5-85f5-4a6b-b12f-0cfb80a42f24 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:02.559648 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98b7ad5-85f5-4a6b-b12f-0cfb80a42f24 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:02.565167 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 492/1000] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Aug 31 05:07:02 2022] GET /identity/v3/endpoints?service_id=fb30a2e41b7847a7950e667b5a35e778&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:07:02.570541 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9c7c98-6a2c-4e2d-afa0-1fef711f2219 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:02.678739 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464640fa-f0b8-4fc4-b9c2-692725104d9e None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:02.678970 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9c7c98-6a2c-4e2d-afa0-1fef711f2219 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:02.679397 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464640fa-f0b8-4fc4-b9c2-692725104d9e None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:02.679397 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464640fa-f0b8-4fc4-b9c2-692725104d9e None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:02.680945 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 493/1001] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:07:02.686453 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe581be-8635-4f99-a7e6-b05e4c219e51 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:02.686737 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe581be-8635-4f99-a7e6-b05e4c219e51 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:02.687008 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe581be-8635-4f99-a7e6-b05e4c219e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:02.705325 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-dfe581be-8635-4f99-a7e6-b05e4c219e51 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:07:02.706547 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-dfe581be-8635-4f99-a7e6-b05e4c219e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:07:02.714602 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddbbb97-8afb-4faf-8361-a83e1c6e88f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:02.715270 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddbbb97-8afb-4faf-8361-a83e1c6e88f8 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:02.715543 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddbbb97-8afb-4faf-8361-a83e1c6e88f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:02.715807 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddbbb97-8afb-4faf-8361-a83e1c6e88f8 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:02.716410 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddbbb97-8afb-4faf-8361-a83e1c6e88f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:02.717772 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddbbb97-8afb-4faf-8361-a83e1c6e88f8 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:02.724292 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 509/1002] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Aug 31 05:07:02 2022] GET /identity/v3/services => generated 1727 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:07:02.784714 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 494/1003] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:07:02 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:07:03.035229 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a99ebf-c7e6-4fc7-99ce-4c1d64fb9fb0 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:03.035415 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a99ebf-c7e6-4fc7-99ce-4c1d64fb9fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:03.035708 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a99ebf-c7e6-4fc7-99ce-4c1d64fb9fb0 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:03.037175 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 510/1004] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:07:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:07:03.042332 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cbe35c-9464-48bd-b8ac-7d2c97b5df4d None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:03.042589 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cbe35c-9464-48bd-b8ac-7d2c97b5df4d None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:03.042885 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cbe35c-9464-48bd-b8ac-7d2c97b5df4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:03.051443 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a99ebf-c7e6-4fc7-99ce-4c1d64fb9fb0 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:03.062976 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-a3cbe35c-9464-48bd-b8ac-7d2c97b5df4d None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:07:03.064509 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a3cbe35c-9464-48bd-b8ac-7d2c97b5df4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:07:03.089102 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:03.136515 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 495/1005] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:07:03 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:07:03.162734 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a99ebf-c7e6-4fc7-99ce-4c1d64fb9fb0 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:03.198999 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343ef65e-700b-49aa-be39-9ef7098931ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:03.199662 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ef65e-700b-49aa-be39-9ef7098931ee None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:03.199939 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ef65e-700b-49aa-be39-9ef7098931ee None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:03.200200 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ef65e-700b-49aa-be39-9ef7098931ee None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:03.200848 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343ef65e-700b-49aa-be39-9ef7098931ee None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:03.202229 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343ef65e-700b-49aa-be39-9ef7098931ee None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:03.226045 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 511/1006] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Aug 31 05:07:03 2022] POST /identity/v3/services => generated 242 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:07:03.405638 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cbe35c-9464-48bd-b8ac-7d2c97b5df4d None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:03.464302 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:03.509639 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cbe35c-9464-48bd-b8ac-7d2c97b5df4d None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:03.550149 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa09746-0afc-494e-831a-7df3bf41960c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:03.550780 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa09746-0afc-494e-831a-7df3bf41960c None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:03.551053 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa09746-0afc-494e-831a-7df3bf41960c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:03.551316 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa09746-0afc-494e-831a-7df3bf41960c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:03.551894 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa09746-0afc-494e-831a-7df3bf41960c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:03.553264 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa09746-0afc-494e-831a-7df3bf41960c None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:03.562173 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-ffa09746-0afc-494e-831a-7df3bf41960c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 05:07:03.562173 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:07:03.562173 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 05:07:03.562173 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 05:07:03.562173 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 05:07:03.562173 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 05:07:03.562173 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 05:07:03.562173 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:07:03.562173 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 05:07:03.562173 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:07:03.562173 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 05:07:03.562173 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:07:03.562173 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:07:03.562173 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:07:03.562173 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:07:03.562173 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:07:03.562173 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:07:03.563199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:07:03.563199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:07:03.563199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:07:03.563199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:07:03.563199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:07:03.563199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:07:03.563199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 05:07:03.563199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 05:07:03.563199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 05:07:03.563199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 05:07:03.563199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:07:03.563199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:07:03.563199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:07:03.563199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:07:03.563199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:07:03.563199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:07:03.564048 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:07:03.564048 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:07:03.564048 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:07:03.564048 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:07:03.564048 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:07:03.564048 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:07:03.564048 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:07:03.564048 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:07:03.564048 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:07:03.564048 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:07:03.564048 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 05:07:03.564048 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:07:03.564048 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 05:07:03.564048 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:07:03.564048 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 496/1007] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Aug 31 05:07:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:07:03.570838 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343ef65e-700b-49aa-be39-9ef7098931ee None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:03.669747 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:03.678919 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343ef65e-700b-49aa-be39-9ef7098931ee None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:03.714562 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c210aaf1-2fd7-4d98-82a2-3755e40d1e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:03.715269 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c210aaf1-2fd7-4d98-82a2-3755e40d1e43 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:03.715508 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c210aaf1-2fd7-4d98-82a2-3755e40d1e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:03.715778 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c210aaf1-2fd7-4d98-82a2-3755e40d1e43 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:03.716416 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c210aaf1-2fd7-4d98-82a2-3755e40d1e43 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:03.716716 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c210aaf1-2fd7-4d98-82a2-3755e40d1e43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:03.718099 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c210aaf1-2fd7-4d98-82a2-3755e40d1e43 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:03.723303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 512/1008] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Aug 31 05:07:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:03.729069 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa09746-0afc-494e-831a-7df3bf41960c None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:03.834021 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa09746-0afc-494e-831a-7df3bf41960c None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:03.850670 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:03.870354 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063c0cda-1f5c-4072-9dab-98ce3529c97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:03.870960 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c0cda-1f5c-4072-9dab-98ce3529c97c None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:03.871231 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c0cda-1f5c-4072-9dab-98ce3529c97c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:03.871527 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c0cda-1f5c-4072-9dab-98ce3529c97c None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:03.872147 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063c0cda-1f5c-4072-9dab-98ce3529c97c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:03.872448 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063c0cda-1f5c-4072-9dab-98ce3529c97c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:03.873818 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063c0cda-1f5c-4072-9dab-98ce3529c97c None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:03.878937 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 497/1009] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Aug 31 05:07:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:07:03.884510 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c210aaf1-2fd7-4d98-82a2-3755e40d1e43 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:03.987165 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c210aaf1-2fd7-4d98-82a2-3755e40d1e43 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:04.023553 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7df1a6-a665-440e-a9fa-b92e00cd2db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:04.024154 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7df1a6-a665-440e-a9fa-b92e00cd2db5 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:04.024417 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7df1a6-a665-440e-a9fa-b92e00cd2db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:04.024687 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7df1a6-a665-440e-a9fa-b92e00cd2db5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:04.025361 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7df1a6-a665-440e-a9fa-b92e00cd2db5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:04.026749 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7df1a6-a665-440e-a9fa-b92e00cd2db5 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:04.044857 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 513/1010] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:07:03 2022] POST /identity/v3/endpoints => generated 325 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:07:04.636743 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757e60e-1333-4f58-be6e-4911262caaf1 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:04.637404 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757e60e-1333-4f58-be6e-4911262caaf1 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:04.637404 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757e60e-1333-4f58-be6e-4911262caaf1 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:04.638747 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 498/1011] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:07:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:07:04.646467 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2def1e3-72e5-400c-aad1-2af11e23c10f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:04.646748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2def1e3-72e5-400c-aad1-2af11e23c10f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:04.647021 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2def1e3-72e5-400c-aad1-2af11e23c10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:04.665501 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d2def1e3-72e5-400c-aad1-2af11e23c10f None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:07:04.666714 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d2def1e3-72e5-400c-aad1-2af11e23c10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:07:04.681046 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:04.745347 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 514/1012] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:07:04 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:07:05.006722 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757e60e-1333-4f58-be6e-4911262caaf1 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:05.116831 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757e60e-1333-4f58-be6e-4911262caaf1 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:05.132370 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957815f-05aa-4581-a954-62f40a82e41e None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:05.132973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957815f-05aa-4581-a954-62f40a82e41e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:05.133483 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957815f-05aa-4581-a954-62f40a82e41e None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:05.135071 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 515/1013] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:07:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 05:07:05.140072 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e66aeb2-44f4-47b6-9c19-162951b59e6f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:05.140569 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e66aeb2-44f4-47b6-9c19-162951b59e6f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:05.141044 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e66aeb2-44f4-47b6-9c19-162951b59e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:05.153828 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275750ea-db23-40de-832a-e56d2da061a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:05.154475 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275750ea-db23-40de-832a-e56d2da061a6 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:05.154749 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275750ea-db23-40de-832a-e56d2da061a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:05.155017 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275750ea-db23-40de-832a-e56d2da061a6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:05.155607 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275750ea-db23-40de-832a-e56d2da061a6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:05.156987 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275750ea-db23-40de-832a-e56d2da061a6 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:05.160126 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-2e66aeb2-44f4-47b6-9c19-162951b59e6f None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:07:05.161293 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2e66aeb2-44f4-47b6-9c19-162951b59e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:07:05.165863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-275750ea-db23-40de-832a-e56d2da061a6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 05:07:05.165863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:07:05.165863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 05:07:05.165863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 05:07:05.165863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 05:07:05.165863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 05:07:05.165863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 05:07:05.165863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:07:05.165863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 05:07:05.165863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:07:05.165863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 05:07:05.165863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:07:05.165863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:07:05.165863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:07:05.165863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:07:05.165863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:07:05.165863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:07:05.166870 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:07:05.166870 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:07:05.166870 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:07:05.166870 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:07:05.166870 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:07:05.166870 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:07:05.166870 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 05:07:05.166870 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 05:07:05.166870 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 05:07:05.166870 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 05:07:05.166870 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:07:05.166870 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:07:05.166870 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:07:05.166870 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:07:05.166870 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:07:05.166870 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:07:05.168304 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:07:05.168304 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:07:05.168304 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:07:05.168304 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:07:05.168304 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:07:05.168304 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:07:05.168304 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:07:05.168304 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:07:05.168304 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:07:05.168304 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:07:05.168304 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 05:07:05.168304 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:07:05.168304 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 05:07:05.168304 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:07:05.168304 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 499/1014] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:07:05 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:07:05.172761 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275750ea-db23-40de-832a-e56d2da061a6 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:05.223061 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:05.237255 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 516/1015] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:07:05 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 05:07:05.279557 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275750ea-db23-40de-832a-e56d2da061a6 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:05.314857 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108689ae-4a8f-45b7-b8c0-a0c071142f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:05.315458 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108689ae-4a8f-45b7-b8c0-a0c071142f3e None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:05.315730 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108689ae-4a8f-45b7-b8c0-a0c071142f3e None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:05.315996 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108689ae-4a8f-45b7-b8c0-a0c071142f3e None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:05.316616 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108689ae-4a8f-45b7-b8c0-a0c071142f3e None admin] RBAC: Adding query filter params (name=compute) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:05.316914 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108689ae-4a8f-45b7-b8c0-a0c071142f3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:05.318314 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108689ae-4a8f-45b7-b8c0-a0c071142f3e None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:05.323441 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 500/1016] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Aug 31 05:07:05 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:07:05.329059 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e66aeb2-44f4-47b6-9c19-162951b59e6f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:05.411368 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:05.435755 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e66aeb2-44f4-47b6-9c19-162951b59e6f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:05.470276 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b1143a-3910-4233-91d9-2b04495fde57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:05.470887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b1143a-3910-4233-91d9-2b04495fde57 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:05.471162 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b1143a-3910-4233-91d9-2b04495fde57 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:05.471434 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b1143a-3910-4233-91d9-2b04495fde57 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:05.472064 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b1143a-3910-4233-91d9-2b04495fde57 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:05.472362 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b1143a-3910-4233-91d9-2b04495fde57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:05.473741 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b1143a-3910-4233-91d9-2b04495fde57 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:05.478999 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 517/1017] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Aug 31 05:07:05 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:07:05.484824 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108689ae-4a8f-45b7-b8c0-a0c071142f3e None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:05.498967 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b1143a-3910-4233-91d9-2b04495fde57 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:05.598744 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108689ae-4a8f-45b7-b8c0-a0c071142f3e None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:05.607013 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b1143a-3910-4233-91d9-2b04495fde57 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:05.608920 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:05.635421 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5010e97-2049-4819-84b7-09243a2c6b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:05.636034 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5010e97-2049-4819-84b7-09243a2c6b80 None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:05.636316 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5010e97-2049-4819-84b7-09243a2c6b80 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:05.636590 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5010e97-2049-4819-84b7-09243a2c6b80 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:05.637308 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5010e97-2049-4819-84b7-09243a2c6b80 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:05.638722 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5010e97-2049-4819-84b7-09243a2c6b80 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:05.651134 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e5a1d8-7aa4-412c-b68f-5ebf54d75e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:05.652067 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5a1d8-7aa4-412c-b68f-5ebf54d75e95 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:05.652614 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5a1d8-7aa4-412c-b68f-5ebf54d75e95 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:05.653071 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5a1d8-7aa4-412c-b68f-5ebf54d75e95 None admin] PATH_INFO: `/v3/services/image` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:05.653891 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e5a1d8-7aa4-412c-b68f-5ebf54d75e95 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:05.655522 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e5a1d8-7aa4-412c-b68f-5ebf54d75e95 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:05.661062 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 501/1018] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:07:05 2022] POST /identity/v3/endpoints => generated 334 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:07:05.664908 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-c2e5a1d8-7aa4-412c-b68f-5ebf54d75e95 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 05:07:05.664908 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:07:05.664908 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 05:07:05.664908 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 05:07:05.664908 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 05:07:05.664908 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 05:07:05.664908 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 05:07:05.664908 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:07:05.664908 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 05:07:05.664908 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:07:05.664908 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 05:07:05.664908 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:07:05.664908 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:07:05.664908 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:07:05.664908 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:07:05.664908 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:07:05.664908 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:07:05.666147 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:07:05.666147 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:07:05.666147 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:07:05.666147 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:07:05.666147 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:07:05.666147 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:07:05.666147 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 05:07:05.666147 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 05:07:05.666147 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 05:07:05.666147 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 05:07:05.666147 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:07:05.666147 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:07:05.666147 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:07:05.666147 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:07:05.666147 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:07:05.666147 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:07:05.667194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:07:05.667194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:07:05.667194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:07:05.667194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:07:05.667194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:07:05.667194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:07:05.667194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:07:05.667194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:07:05.667194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:07:05.667194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:07:05.667194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 05:07:05.667194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:07:05.667194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 05:07:05.667194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:07:05.669892 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 518/1019] 10.222.0.36 () {56 vars in 1105 bytes} [Wed Aug 31 05:07:05 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:07:05.675900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5010e97-2049-4819-84b7-09243a2c6b80 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:05.811622 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5010e97-2049-4819-84b7-09243a2c6b80 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:05.852218 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd905d3-a679-4f7c-8228-b9fcc870875b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:05.852755 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd905d3-a679-4f7c-8228-b9fcc870875b None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:05.853028 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd905d3-a679-4f7c-8228-b9fcc870875b None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:05.853295 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd905d3-a679-4f7c-8228-b9fcc870875b None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:05.853940 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd905d3-a679-4f7c-8228-b9fcc870875b None admin] RBAC: Adding query filter params (name=image) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:05.854263 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd905d3-a679-4f7c-8228-b9fcc870875b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:05.855642 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd905d3-a679-4f7c-8228-b9fcc870875b None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:05.860846 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 502/1020] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:07:05 2022] GET /identity/v3/services?name=image => generated 122 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:05.866845 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e5a1d8-7aa4-412c-b68f-5ebf54d75e95 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:05.871059 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:05.939876 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb974f9-24bc-432d-b0b9-1cf0c19032d1 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:05.940538 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb974f9-24bc-432d-b0b9-1cf0c19032d1 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:05.941069 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb974f9-24bc-432d-b0b9-1cf0c19032d1 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:05.942818 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 503/1021] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:07:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:07:05.949711 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f76c94-2748-4e02-97be-0ae74dba0634 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:05.949994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f76c94-2748-4e02-97be-0ae74dba0634 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:05.950299 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f76c94-2748-4e02-97be-0ae74dba0634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:05.971543 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-b0f76c94-2748-4e02-97be-0ae74dba0634 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:07:05.972837 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b0f76c94-2748-4e02-97be-0ae74dba0634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:07:05.985952 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e5a1d8-7aa4-412c-b68f-5ebf54d75e95 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:05.995340 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:06.023716 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb34ed-0e49-45e4-a1a5-8cdf87f3683e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:06.024676 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb34ed-0e49-45e4-a1a5-8cdf87f3683e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:06.024975 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb34ed-0e49-45e4-a1a5-8cdf87f3683e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:06.025246 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb34ed-0e49-45e4-a1a5-8cdf87f3683e None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:06.025883 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eb34ed-0e49-45e4-a1a5-8cdf87f3683e None admin] RBAC: Adding query filter params (type=image) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:06.026394 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eb34ed-0e49-45e4-a1a5-8cdf87f3683e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:06.027623 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eb34ed-0e49-45e4-a1a5-8cdf87f3683e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:06.033055 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 519/1022] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:07:05 2022] GET /identity/v3/services?type=image => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:06.038850 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb34ed-0e49-45e4-a1a5-8cdf87f3683e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:06.057646 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 504/1023] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:07:05 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:07:06.112672 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:06.151735 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb34ed-0e49-45e4-a1a5-8cdf87f3683e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:06.186731 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d394a5-0a3b-4bb5-b356-fae08c390c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:06.187342 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d394a5-0a3b-4bb5-b356-fae08c390c49 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:06.187617 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d394a5-0a3b-4bb5-b356-fae08c390c49 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:06.187883 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d394a5-0a3b-4bb5-b356-fae08c390c49 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:06.188612 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d394a5-0a3b-4bb5-b356-fae08c390c49 None admin] RBAC: Adding query filter params (interface=public, service_id=455fb5d7fdb04da8aa61635a2623ad72, region_id=RegionOne) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:06.188916 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d394a5-0a3b-4bb5-b356-fae08c390c49 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:06.190308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d394a5-0a3b-4bb5-b356-fae08c390c49 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:06.195728 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 520/1024] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Aug 31 05:07:06 2022] GET /identity/v3/endpoints?service_id=455fb5d7fdb04da8aa61635a2623ad72&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:07:06.201428 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f76c94-2748-4e02-97be-0ae74dba0634 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:06.305182 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f76c94-2748-4e02-97be-0ae74dba0634 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:06.327969 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d394a5-0a3b-4bb5-b356-fae08c390c49 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:06.341443 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33bdc56-1d27-4b6e-8d00-15b52d0112e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:06.342072 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33bdc56-1d27-4b6e-8d00-15b52d0112e1 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:06.342360 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33bdc56-1d27-4b6e-8d00-15b52d0112e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:06.342616 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33bdc56-1d27-4b6e-8d00-15b52d0112e1 None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:06.343240 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33bdc56-1d27-4b6e-8d00-15b52d0112e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:06.344624 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33bdc56-1d27-4b6e-8d00-15b52d0112e1 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:06.352627 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 505/1025] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Aug 31 05:07:06 2022] GET /identity/v3/services => generated 1957 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:07:06.448686 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d394a5-0a3b-4bb5-b356-fae08c390c49 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:06.480697 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:06.488271 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741167e-52a9-4abe-aa67-68ffa46fb527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:06.488921 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741167e-52a9-4abe-aa67-68ffa46fb527 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:06.489193 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741167e-52a9-4abe-aa67-68ffa46fb527 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:06.489455 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741167e-52a9-4abe-aa67-68ffa46fb527 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:06.490584 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c741167e-52a9-4abe-aa67-68ffa46fb527 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:06.492032 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c741167e-52a9-4abe-aa67-68ffa46fb527 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:06.498032 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 521/1026] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:07:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:07:06.500432 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33bdc56-1d27-4b6e-8d00-15b52d0112e1 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:06.522323 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:06.607579 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33bdc56-1d27-4b6e-8d00-15b52d0112e1 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:06.656388 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1774e37-6224-4fed-acc4-2e91eadfb868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:06.656999 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1774e37-6224-4fed-acc4-2e91eadfb868 None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:06.657245 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1774e37-6224-4fed-acc4-2e91eadfb868 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:06.657523 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1774e37-6224-4fed-acc4-2e91eadfb868 None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:06.658221 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1774e37-6224-4fed-acc4-2e91eadfb868 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:06.659651 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1774e37-6224-4fed-acc4-2e91eadfb868 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:06.694050 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 506/1027] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Aug 31 05:07:06 2022] POST /identity/v3/projects => generated 311 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:07:07.658203 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a21a9f-ebcf-4337-930d-3df7769173d2 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:07.658690 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a21a9f-ebcf-4337-930d-3df7769173d2 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:07.658796 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a21a9f-ebcf-4337-930d-3df7769173d2 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:07.660223 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 522/1028] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:07:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:07:07.665130 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90b2b5-72fd-4435-838a-1b0eaa67a680 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:07.665509 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90b2b5-72fd-4435-838a-1b0eaa67a680 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:07.665686 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90b2b5-72fd-4435-838a-1b0eaa67a680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:07.684367 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-0f90b2b5-72fd-4435-838a-1b0eaa67a680 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:07:07.685538 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-0f90b2b5-72fd-4435-838a-1b0eaa67a680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:07:07.750198 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:07.761484 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 507/1029] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:07:07 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:07:08.014883 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a21a9f-ebcf-4337-930d-3df7769173d2 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:08.124908 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a21a9f-ebcf-4337-930d-3df7769173d2 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:08.158556 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611eb979-f861-43f9-9b66-1d2622097859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:08.159177 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611eb979-f861-43f9-9b66-1d2622097859 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:08.159450 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611eb979-f861-43f9-9b66-1d2622097859 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:08.159715 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611eb979-f861-43f9-9b66-1d2622097859 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:08.170331 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611eb979-f861-43f9-9b66-1d2622097859 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:07:08.174982 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-611eb979-f861-43f9-9b66-1d2622097859 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 05:07:08.174982 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:07:08.174982 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:07:08.174982 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:07:08.174982 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:07:08.174982 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:07:08.174982 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:07:08.174982 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:07:08.174982 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:07:08.174982 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:07:08.174982 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:07:08.174982 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:07:08.174982 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:07:08.174982 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:07:08.174982 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:07:08.174982 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:07:08.175895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:07:08.175895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:07:08.175895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:07:08.175895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:07:08.175895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:07:08.175895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:07:08.175895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:07:08.175895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:07:08.175895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:07:08.175895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:07:08.175895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:07:08.175895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:07:08.175895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:07:08.175895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:07:08.175895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:07:08.175895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:07:08.176757 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:07:08.176757 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:07:08.176757 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:07:08.176757 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:07:08.176757 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:07:08.176757 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:07:08.176757 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:07:08.176757 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:07:08.176757 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:07:08.176757 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:07:08.176757 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 05:07:08.176757 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:07:08.184178 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 523/1030] 10.222.0.36 () {56 vars in 1120 bytes} [Wed Aug 31 05:07:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:07:08.184318 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90b2b5-72fd-4435-838a-1b0eaa67a680 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:08.266161 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd138b97-5c20-476c-8bc2-e267bff582bf None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:08.266368 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd138b97-5c20-476c-8bc2-e267bff582bf None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:08.266647 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd138b97-5c20-476c-8bc2-e267bff582bf None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:08.268147 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 524/1031] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:07:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 05:07:08.272895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3f419-d019-4c11-9d08-f176c388e4b8 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:08.273150 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3f419-d019-4c11-9d08-f176c388e4b8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:08.273445 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3f419-d019-4c11-9d08-f176c388e4b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:08.290658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90b2b5-72fd-4435-838a-1b0eaa67a680 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:08.292381 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-3ea3f419-d019-4c11-9d08-f176c388e4b8 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:07:08.293955 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3ea3f419-d019-4c11-9d08-f176c388e4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:07:08.330184 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ada949-dd3c-415a-9cc5-480ed90f0a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:08.330782 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ada949-dd3c-415a-9cc5-480ed90f0a20 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:08.331048 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ada949-dd3c-415a-9cc5-480ed90f0a20 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:08.331310 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ada949-dd3c-415a-9cc5-480ed90f0a20 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:08.331968 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ada949-dd3c-415a-9cc5-480ed90f0a20 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:08.332263 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ada949-dd3c-415a-9cc5-480ed90f0a20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:08.333635 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ada949-dd3c-415a-9cc5-480ed90f0a20 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:08.346269 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 508/1032] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Aug 31 05:07:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:08.347174 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ada949-dd3c-415a-9cc5-480ed90f0a20 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:08.354531 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:08.369905 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 525/1033] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:07:08 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 05:07:08.412234 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:08.453314 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ada949-dd3c-415a-9cc5-480ed90f0a20 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:08.488192 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fcc39f-cc23-45f0-8082-558b8d57c8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:08.488821 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fcc39f-cc23-45f0-8082-558b8d57c8b7 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:08.489099 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fcc39f-cc23-45f0-8082-558b8d57c8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:08.489333 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fcc39f-cc23-45f0-8082-558b8d57c8b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:08.500517 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fcc39f-cc23-45f0-8082-558b8d57c8b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:08.502018 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fcc39f-cc23-45f0-8082-558b8d57c8b7 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:08.516068 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-b3fcc39f-cc23-45f0-8082-558b8d57c8b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:07:08.516068 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:07:08.516068 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:07:08.516068 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:07:08.516068 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:07:08.516068 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:07:08.516068 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:07:08.516068 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:07:08.516068 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:07:08.516068 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:07:08.516068 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:07:08.516068 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:07:08.516068 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:07:08.516068 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:07:08.516068 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:07:08.516068 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:07:08.517062 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:07:08.517062 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:07:08.517062 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:07:08.517062 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:07:08.517062 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:07:08.517062 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:07:08.517062 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:07:08.517062 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:07:08.517062 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:07:08.517062 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:07:08.517062 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:07:08.517062 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:07:08.517062 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:07:08.517062 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:07:08.517062 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:07:08.517062 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:07:08.517062 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:07:08.517971 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:07:08.517971 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:07:08.517971 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:07:08.522252 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 509/1034] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:07:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:07:08.523403 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea3f419-d019-4c11-9d08-f176c388e4b8 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:08.627379 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea3f419-d019-4c11-9d08-f176c388e4b8 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:08.631058 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1cd348-2265-4713-ae99-42960992c01f None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:08.631338 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1cd348-2265-4713-ae99-42960992c01f None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:08.631607 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1cd348-2265-4713-ae99-42960992c01f None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:08.632936 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 510/1035] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:07:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 05:07:08.635543 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1cd348-2265-4713-ae99-42960992c01f None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:08.667717 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c813d4-21bd-46d1-a942-8df2d0255b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:08.668375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c813d4-21bd-46d1-a942-8df2d0255b22 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:08.668634 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c813d4-21bd-46d1-a942-8df2d0255b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:08.668899 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c813d4-21bd-46d1-a942-8df2d0255b22 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:08.669532 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c813d4-21bd-46d1-a942-8df2d0255b22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:08.669826 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c813d4-21bd-46d1-a942-8df2d0255b22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:08.671205 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c813d4-21bd-46d1-a942-8df2d0255b22 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:08.682327 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 526/1036] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:07:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:08.683468 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac24c65-8727-4bf6-8185-885015111d08 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:08.683844 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac24c65-8727-4bf6-8185-885015111d08 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:08.684109 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac24c65-8727-4bf6-8185-885015111d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:08.701418 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-bac24c65-8727-4bf6-8185-885015111d08 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:07:08.702616 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-bac24c65-8727-4bf6-8185-885015111d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:07:08.744657 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:08.745779 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1cd348-2265-4713-ae99-42960992c01f None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:08.760139 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:08.767561 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 527/1037] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:07:08 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:07:08.768312 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac24c65-8727-4bf6-8185-885015111d08 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:08.783264 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb0ca3-13d7-4333-81cd-e60e86771e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:08.783898 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb0ca3-13d7-4333-81cd-e60e86771e8c None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:08.784172 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb0ca3-13d7-4333-81cd-e60e86771e8c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:08.784436 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb0ca3-13d7-4333-81cd-e60e86771e8c None admin] PATH_INFO: `/v3/services/image` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:08.785023 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb0ca3-13d7-4333-81cd-e60e86771e8c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:08.786440 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb0ca3-13d7-4333-81cd-e60e86771e8c None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:08.795765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-25cb0ca3-13d7-4333-81cd-e60e86771e8c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 05:07:08.795765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:07:08.795765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 05:07:08.795765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 05:07:08.795765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 05:07:08.795765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 05:07:08.795765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 05:07:08.795765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:07:08.795765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 05:07:08.795765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:07:08.795765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 05:07:08.795765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:07:08.795765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:07:08.795765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:07:08.795765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:07:08.795765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:07:08.795765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:07:08.796910 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:07:08.796910 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:07:08.796910 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:07:08.796910 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:07:08.796910 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:07:08.796910 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:07:08.796910 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 05:07:08.796910 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 05:07:08.796910 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 05:07:08.796910 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 05:07:08.796910 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:07:08.796910 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:07:08.796910 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:07:08.796910 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:07:08.796910 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:07:08.796910 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:07:08.798529 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:07:08.798529 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:07:08.798529 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:07:08.798529 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:07:08.798529 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:07:08.798529 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:07:08.798529 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:07:08.798529 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:07:08.798529 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:07:08.798529 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:07:08.798529 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 05:07:08.798529 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:07:08.798529 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 05:07:08.798529 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:07:08.798529 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 511/1038] 10.222.0.36 () {56 vars in 1105 bytes} [Wed Aug 31 05:07:08 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:07:08.803056 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb0ca3-13d7-4333-81cd-e60e86771e8c None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:08.872934 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac24c65-8727-4bf6-8185-885015111d08 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:08.908942 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fa5345-91ef-4f27-bd9e-da9d74d500ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:08.909572 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa5345-91ef-4f27-bd9e-da9d74d500ac None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:08.909852 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa5345-91ef-4f27-bd9e-da9d74d500ac None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:08.910151 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa5345-91ef-4f27-bd9e-da9d74d500ac None admin] PATH_INFO: `/v3/users/nova` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:08.918782 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fa5345-91ef-4f27-bd9e-da9d74d500ac None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:08.919142 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb0ca3-13d7-4333-81cd-e60e86771e8c None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:08.920481 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fa5345-91ef-4f27-bd9e-da9d74d500ac None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:08.932181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-c9fa5345-91ef-4f27-bd9e-da9d74d500ac None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 05:07:08.932181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:07:08.932181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:07:08.932181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:07:08.932181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:07:08.932181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:07:08.932181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:07:08.932181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:07:08.932181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:07:08.932181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:07:08.932181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:07:08.932181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:07:08.932181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 05:07:08.932181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 05:07:08.932181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 05:07:08.932181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 05:07:08.933145 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:07:08.933145 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:07:08.933145 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 05:07:08.933145 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:07:08.933145 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 05:07:08.933145 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:07:08.933145 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:07:08.933145 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:07:08.933145 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:07:08.933145 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:07:08.933145 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:07:08.933145 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:07:08.933145 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:07:08.933145 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:07:08.933145 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:07:08.933145 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:07:08.933145 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:07:08.933145 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:07:08.934305 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:07:08.934305 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:07:08.934305 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 05:07:08.934305 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 05:07:08.934305 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 05:07:08.934305 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 05:07:08.934305 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 05:07:08.934305 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 05:07:08.934305 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 05:07:08.934305 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:07:08.934305 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 528/1039] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Aug 31 05:07:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:07:08.940427 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fa5345-91ef-4f27-bd9e-da9d74d500ac None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:08.959180 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc201d70-00d4-41ab-9e0b-fbf47c4235d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:08.959760 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc201d70-00d4-41ab-9e0b-fbf47c4235d4 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:08.960038 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc201d70-00d4-41ab-9e0b-fbf47c4235d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:08.960308 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc201d70-00d4-41ab-9e0b-fbf47c4235d4 None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:08.961207 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc201d70-00d4-41ab-9e0b-fbf47c4235d4 None admin] RBAC: Adding query filter params (name=image) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:08.961207 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc201d70-00d4-41ab-9e0b-fbf47c4235d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:08.962783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc201d70-00d4-41ab-9e0b-fbf47c4235d4 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:08.967667 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 512/1040] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:07:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:08.973343 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc201d70-00d4-41ab-9e0b-fbf47c4235d4 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:08.989235 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:09.020462 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:09.054260 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fa5345-91ef-4f27-bd9e-da9d74d500ac None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:09.075596 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc201d70-00d4-41ab-9e0b-fbf47c4235d4 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:09.087563 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4034a7-32ae-4694-bc67-1be0546650f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:09.088171 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4034a7-32ae-4694-bc67-1be0546650f6 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:09.088444 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4034a7-32ae-4694-bc67-1be0546650f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:09.088712 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4034a7-32ae-4694-bc67-1be0546650f6 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:09.089410 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4034a7-32ae-4694-bc67-1be0546650f6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:09.089710 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4034a7-32ae-4694-bc67-1be0546650f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:09.091222 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4034a7-32ae-4694-bc67-1be0546650f6 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:09.108005 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 529/1041] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Aug 31 05:07:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:07:09.108878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706aa0e-6031-49a3-af27-f2f64bf7a127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:09.109481 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706aa0e-6031-49a3-af27-f2f64bf7a127 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:09.109751 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706aa0e-6031-49a3-af27-f2f64bf7a127 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:09.110036 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706aa0e-6031-49a3-af27-f2f64bf7a127 None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:09.110661 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6706aa0e-6031-49a3-af27-f2f64bf7a127 None admin] RBAC: Adding query filter params (type=image) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:09.110879 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4034a7-32ae-4694-bc67-1be0546650f6 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:09.110975 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6706aa0e-6031-49a3-af27-f2f64bf7a127 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:09.112303 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6706aa0e-6031-49a3-af27-f2f64bf7a127 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:09.121956 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 513/1042] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:07:08 2022] GET /identity/v3/services?type=image => generated 350 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:07:09.121956 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706aa0e-6031-49a3-af27-f2f64bf7a127 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:09.202738 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:09.209018 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4034a7-32ae-4694-bc67-1be0546650f6 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:09.219331 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706aa0e-6031-49a3-af27-f2f64bf7a127 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:09.226326 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:09.244010 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4157e72-3efe-4232-b0a9-1ecf1e41af02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:09.244628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4157e72-3efe-4232-b0a9-1ecf1e41af02 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:09.244902 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4157e72-3efe-4232-b0a9-1ecf1e41af02 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:09.245168 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4157e72-3efe-4232-b0a9-1ecf1e41af02 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:09.247942 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4157e72-3efe-4232-b0a9-1ecf1e41af02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:09.247942 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4157e72-3efe-4232-b0a9-1ecf1e41af02 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:09.249298 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 530/1043] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:07:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:09.250043 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4157e72-3efe-4232-b0a9-1ecf1e41af02 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:09.258359 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f00b9ae-bee1-4b0f-b4f9-7517f5027dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:09.258964 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00b9ae-bee1-4b0f-b4f9-7517f5027dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:09.259233 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00b9ae-bee1-4b0f-b4f9-7517f5027dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:09.259493 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00b9ae-bee1-4b0f-b4f9-7517f5027dc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:09.329985 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f00b9ae-bee1-4b0f-b4f9-7517f5027dc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:09.331377 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f00b9ae-bee1-4b0f-b4f9-7517f5027dc1 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:09.349995 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4157e72-3efe-4232-b0a9-1ecf1e41af02 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:09.384298 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ead4d4-3a37-4a72-9957-c541cfb6cc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:09.384903 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ead4d4-3a37-4a72-9957-c541cfb6cc68 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:09.385177 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ead4d4-3a37-4a72-9957-c541cfb6cc68 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:09.385444 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ead4d4-3a37-4a72-9957-c541cfb6cc68 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:09.386137 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ead4d4-3a37-4a72-9957-c541cfb6cc68 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:09.387572 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ead4d4-3a37-4a72-9957-c541cfb6cc68 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:09.407882 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 531/1044] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:07:09 2022] POST /identity/v3/endpoints => generated 327 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:07:09.408822 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ead4d4-3a37-4a72-9957-c541cfb6cc68 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:09.433477 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 514/1045] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:07:09 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 05:07:09.433874 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:09.439671 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f00b9ae-bee1-4b0f-b4f9-7517f5027dc1 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:09.450060 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:09.532293 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ead4d4-3a37-4a72-9957-c541cfb6cc68 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:09.557910 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f00b9ae-bee1-4b0f-b4f9-7517f5027dc1 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:09.570101 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c9d8c-453c-475a-9323-7da3fa590f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:09.571118 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c9d8c-453c-475a-9323-7da3fa590f3f None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:09.571118 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c9d8c-453c-475a-9323-7da3fa590f3f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:09.571304 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c9d8c-453c-475a-9323-7da3fa590f3f None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:09.571881 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79c9d8c-453c-475a-9323-7da3fa590f3f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:09.573904 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79c9d8c-453c-475a-9323-7da3fa590f3f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:09.597605 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 532/1046] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Aug 31 05:07:09 2022] POST /identity/v3/users => generated 285 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:07:09.597783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e1f05d-7486-4a50-b5ea-a284f44ded2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:09.598183 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1f05d-7486-4a50-b5ea-a284f44ded2b None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:09.598394 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1f05d-7486-4a50-b5ea-a284f44ded2b None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:09.598659 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1f05d-7486-4a50-b5ea-a284f44ded2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:09.616444 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e1f05d-7486-4a50-b5ea-a284f44ded2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:09.618285 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e1f05d-7486-4a50-b5ea-a284f44ded2b None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:09.635699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-81e1f05d-7486-4a50-b5ea-a284f44ded2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:07:09.635699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:07:09.635699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:07:09.635699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:07:09.635699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:07:09.635699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:07:09.635699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:07:09.635699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:07:09.635699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:07:09.635699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:07:09.635699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:07:09.635699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:07:09.635699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:07:09.635699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:07:09.635699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:07:09.635699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:07:09.636774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:07:09.636774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:07:09.636774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:07:09.636774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:07:09.636774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:07:09.636774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:07:09.636774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:07:09.636774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:07:09.636774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:07:09.636774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:07:09.636774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:07:09.636774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:07:09.636774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:07:09.636774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:07:09.636774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:07:09.636774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:07:09.636774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:07:09.637679 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:07:09.637679 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:07:09.637679 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:07:09.637679 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 515/1047] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:07:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:07:09.644110 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c9d8c-453c-475a-9323-7da3fa590f3f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:09.729979 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:09.767934 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c9d8c-453c-475a-9323-7da3fa590f3f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:09.802179 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9476b83b-ccb2-4278-b27b-d199407a1ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:09.802788 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476b83b-ccb2-4278-b27b-d199407a1ea7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:09.803071 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476b83b-ccb2-4278-b27b-d199407a1ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:09.803337 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476b83b-ccb2-4278-b27b-d199407a1ea7 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:09.804007 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9476b83b-ccb2-4278-b27b-d199407a1ea7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:09.804308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9476b83b-ccb2-4278-b27b-d199407a1ea7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:09.805674 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9476b83b-ccb2-4278-b27b-d199407a1ea7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:09.818075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 533/1048] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:07:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:09.823967 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e1f05d-7486-4a50-b5ea-a284f44ded2b None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:09.888221 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:09.935488 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e1f05d-7486-4a50-b5ea-a284f44ded2b None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:09.971580 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79293f32-3d24-4678-a96a-4e2a99c6824e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:09.972217 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79293f32-3d24-4678-a96a-4e2a99c6824e None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:09.972454 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79293f32-3d24-4678-a96a-4e2a99c6824e None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:09.972756 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79293f32-3d24-4678-a96a-4e2a99c6824e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:09.980113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79293f32-3d24-4678-a96a-4e2a99c6824e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:09.981651 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79293f32-3d24-4678-a96a-4e2a99c6824e None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:09.993511 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-79293f32-3d24-4678-a96a-4e2a99c6824e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:07:09.993511 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:07:09.993511 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:07:09.993511 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:07:09.993511 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:07:09.993511 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:07:09.993511 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:07:09.993511 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:07:09.993511 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:07:09.993511 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:07:09.993511 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:07:09.993511 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:07:09.993511 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 05:07:09.993511 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 05:07:09.993511 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 05:07:09.994487 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 05:07:09.994487 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:07:09.994487 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:07:09.994487 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:07:09.994487 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:07:09.994487 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:07:09.994487 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:07:09.994487 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:07:09.994487 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:07:09.994487 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:07:09.994487 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:07:09.994487 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:07:09.994487 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:07:09.994487 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:07:09.994487 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:07:09.994487 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:07:09.994487 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:07:09.995406 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 05:07:09.995406 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 05:07:09.995406 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 05:07:09.995406 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 05:07:09.995406 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 05:07:09.995406 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 05:07:09.995406 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:07:09.995406 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:07:09.995406 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 516/1049] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:07:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:07:10.000772 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9476b83b-ccb2-4278-b27b-d199407a1ea7 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:10.104797 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9476b83b-ccb2-4278-b27b-d199407a1ea7 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:10.134796 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:10.141088 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cab487-dfac-4abc-b549-672b9a8f4109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:10.141638 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cab487-dfac-4abc-b549-672b9a8f4109 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:10.141918 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cab487-dfac-4abc-b549-672b9a8f4109 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:10.142215 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cab487-dfac-4abc-b549-672b9a8f4109 None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:10.142910 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cab487-dfac-4abc-b549-672b9a8f4109 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:10.143211 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cab487-dfac-4abc-b549-672b9a8f4109 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:10.144623 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cab487-dfac-4abc-b549-672b9a8f4109 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:10.157106 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 534/1050] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 05:07:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:07:10.162718 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79293f32-3d24-4678-a96a-4e2a99c6824e None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:10.264757 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79293f32-3d24-4678-a96a-4e2a99c6824e None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:10.282829 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:10.301247 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176cf10f-1b90-4b53-a9ce-90183a016661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:10.301847 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176cf10f-1b90-4b53-a9ce-90183a016661 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:10.302230 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176cf10f-1b90-4b53-a9ce-90183a016661 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:10.302499 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176cf10f-1b90-4b53-a9ce-90183a016661 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:10.303227 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176cf10f-1b90-4b53-a9ce-90183a016661 None admin] RBAC: Adding query filter params (role.id=f7c0c9eac46c42179e586dc4c1726356, scope.project.id=e01ce1b1282e44a39d659b87f120995e, user.id=f27665b1574a4db7b7a9771f7d775826) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:10.303537 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176cf10f-1b90-4b53-a9ce-90183a016661 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:10.304973 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176cf10f-1b90-4b53-a9ce-90183a016661 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:10.310753 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 517/1051] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:07:10 2022] GET /identity/v3/role_assignments?user.id=f27665b1574a4db7b7a9771f7d775826&scope.project.id=e01ce1b1282e44a39d659b87f120995e&role.id=f7c0c9eac46c42179e586dc4c1726356 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:11.312208 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21474e95-7b83-43d1-aebe-55a1fae5d920 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:11.312765 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21474e95-7b83-43d1-aebe-55a1fae5d920 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:11.312765 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21474e95-7b83-43d1-aebe-55a1fae5d920 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:11.314182 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 535/1052] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:07:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:07:11.319834 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b536b1-c12d-4c0f-824d-65a8e7bd7ff4 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:11.320284 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b536b1-c12d-4c0f-824d-65a8e7bd7ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:11.320361 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b536b1-c12d-4c0f-824d-65a8e7bd7ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:11.340891 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-65b536b1-c12d-4c0f-824d-65a8e7bd7ff4 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:07:11.342325 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-65b536b1-c12d-4c0f-824d-65a8e7bd7ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:07:11.423301 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 518/1053] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:07:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:07:11.548084 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e84a06-34a0-4e7d-99ea-1edddf170596 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:11.548434 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e84a06-34a0-4e7d-99ea-1edddf170596 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:11.548596 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e84a06-34a0-4e7d-99ea-1edddf170596 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:11.550077 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 536/1054] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:07:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:07:11.555156 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b587674-8b7a-4973-9286-83763fb1aa23 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:11.555441 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b587674-8b7a-4973-9286-83763fb1aa23 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:11.555716 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b587674-8b7a-4973-9286-83763fb1aa23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:11.574361 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-9b587674-8b7a-4973-9286-83763fb1aa23 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:07:11.575454 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-9b587674-8b7a-4973-9286-83763fb1aa23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:07:11.658029 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 519/1055] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:07:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:07:11.692990 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e84a06-34a0-4e7d-99ea-1edddf170596 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:11.801572 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e84a06-34a0-4e7d-99ea-1edddf170596 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:11.835398 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4633a2f-94be-46da-8154-4b50773fb797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:11.835931 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4633a2f-94be-46da-8154-4b50773fb797 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:11.836257 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4633a2f-94be-46da-8154-4b50773fb797 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:11.836522 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4633a2f-94be-46da-8154-4b50773fb797 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:11.896420 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:11.907522 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4633a2f-94be-46da-8154-4b50773fb797 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:11.908975 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4633a2f-94be-46da-8154-4b50773fb797 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:11.909597 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b587674-8b7a-4973-9286-83763fb1aa23 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:11.956223 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:12.017421 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b587674-8b7a-4973-9286-83763fb1aa23 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:12.021430 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 537/1056] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:07:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 05:07:12.029106 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4633a2f-94be-46da-8154-4b50773fb797 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:12.057323 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928695bf-83d3-42de-980d-1905fe3810ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:12.058104 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928695bf-83d3-42de-980d-1905fe3810ac None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:12.058414 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928695bf-83d3-42de-980d-1905fe3810ac None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:12.058704 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928695bf-83d3-42de-980d-1905fe3810ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:12.068825 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928695bf-83d3-42de-980d-1905fe3810ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:07:12.073551 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-928695bf-83d3-42de-980d-1905fe3810ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:07:12.073551 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:07:12.073551 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:07:12.073551 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:07:12.073551 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:07:12.073551 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:07:12.073551 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:07:12.073551 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:07:12.073551 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:07:12.073551 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:07:12.073551 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:07:12.073551 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:07:12.073551 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:07:12.073551 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:07:12.073551 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:07:12.073551 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:07:12.074621 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:07:12.074621 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:07:12.074621 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:07:12.074621 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:07:12.074621 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:07:12.074621 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:07:12.074621 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:07:12.074621 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:07:12.074621 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:07:12.074621 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:07:12.074621 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:07:12.074621 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:07:12.074621 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:07:12.074621 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:07:12.074621 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:07:12.074621 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:07:12.075474 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:07:12.075474 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:07:12.075474 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:07:12.075474 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:07:12.075474 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:07:12.075474 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:07:12.075474 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:07:12.075474 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:07:12.075474 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:07:12.075474 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:07:12.075474 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:07:12.075474 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:07:12.075474 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 520/1057] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 05:07:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:07:12.080832 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928695bf-83d3-42de-980d-1905fe3810ac None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:12.125607 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:12.135937 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4633a2f-94be-46da-8154-4b50773fb797 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:12.171914 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de291b2-87fe-4866-8068-d0aefe644379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:12.172544 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de291b2-87fe-4866-8068-d0aefe644379 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:12.172926 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de291b2-87fe-4866-8068-d0aefe644379 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:12.173193 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de291b2-87fe-4866-8068-d0aefe644379 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:12.182024 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928695bf-83d3-42de-980d-1905fe3810ac None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:12.183527 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de291b2-87fe-4866-8068-d0aefe644379 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:12.183649 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:12.185113 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de291b2-87fe-4866-8068-d0aefe644379 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:12.200631 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-5de291b2-87fe-4866-8068-d0aefe644379 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:07:12.200631 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:07:12.200631 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:07:12.200631 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:07:12.200631 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:07:12.200631 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:07:12.200631 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:07:12.200631 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:07:12.200631 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:07:12.200631 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:07:12.200631 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:07:12.200631 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:07:12.200631 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:07:12.200631 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:07:12.200631 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:07:12.200631 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:07:12.201619 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:07:12.201619 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:07:12.201619 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:07:12.201619 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:07:12.201619 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:07:12.201619 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:07:12.201619 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:07:12.201619 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:07:12.201619 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:07:12.201619 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:07:12.201619 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:07:12.201619 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:07:12.201619 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:07:12.201619 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:07:12.201619 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:07:12.201619 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:07:12.201619 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:07:12.202878 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:07:12.202878 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:07:12.202878 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:07:12.202878 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 538/1058] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:07:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:07:12.202878 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e798fd15-f6af-41d5-9cca-006e7976e085 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:12.203174 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e798fd15-f6af-41d5-9cca-006e7976e085 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:12.203430 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e798fd15-f6af-41d5-9cca-006e7976e085 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:12.204670 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 539/1059] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:07:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:07:12.207874 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e798fd15-f6af-41d5-9cca-006e7976e085 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:12.221613 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a07e12b-5f43-4f62-b83a-ad48ae2db010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:12.222250 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07e12b-5f43-4f62-b83a-ad48ae2db010 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:12.222528 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07e12b-5f43-4f62-b83a-ad48ae2db010 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:12.222796 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07e12b-5f43-4f62-b83a-ad48ae2db010 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:12.223455 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a07e12b-5f43-4f62-b83a-ad48ae2db010 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:12.223750 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a07e12b-5f43-4f62-b83a-ad48ae2db010 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:12.225126 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a07e12b-5f43-4f62-b83a-ad48ae2db010 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:12.233802 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 521/1060] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:07:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:07:12.234956 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40588048-0cc3-4eb3-89ca-6c2c0e0b1ad9 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:12.235236 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40588048-0cc3-4eb3-89ca-6c2c0e0b1ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:12.235500 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40588048-0cc3-4eb3-89ca-6c2c0e0b1ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:12.252990 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-40588048-0cc3-4eb3-89ca-6c2c0e0b1ad9 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:07:12.254207 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-40588048-0cc3-4eb3-89ca-6c2c0e0b1ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:07:12.308734 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e798fd15-f6af-41d5-9cca-006e7976e085 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:12.321704 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 522/1061] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:07:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:07:12.322564 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40588048-0cc3-4eb3-89ca-6c2c0e0b1ad9 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:12.342292 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad3ad9d-b25a-40aa-a2a7-3e5b3077df84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:12.342905 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3ad9d-b25a-40aa-a2a7-3e5b3077df84 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:12.343183 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3ad9d-b25a-40aa-a2a7-3e5b3077df84 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:12.343456 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3ad9d-b25a-40aa-a2a7-3e5b3077df84 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:12.344098 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad3ad9d-b25a-40aa-a2a7-3e5b3077df84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:12.344398 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad3ad9d-b25a-40aa-a2a7-3e5b3077df84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:12.345770 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad3ad9d-b25a-40aa-a2a7-3e5b3077df84 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:12.357576 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 540/1062] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:07:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:07:12.436538 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:12.445108 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40588048-0cc3-4eb3-89ca-6c2c0e0b1ad9 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:12.481888 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3cc7d6-4212-4bff-9d49-d16f14b41488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:12.482573 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3cc7d6-4212-4bff-9d49-d16f14b41488 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:12.482851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3cc7d6-4212-4bff-9d49-d16f14b41488 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:12.483119 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3cc7d6-4212-4bff-9d49-d16f14b41488 None admin] PATH_INFO: `/v3/users/b7c39d57989c43ec88dc094c22a2cdf2` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:12.498268 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3cc7d6-4212-4bff-9d49-d16f14b41488 None admin] RBAC: Authorizing `identity:get_user(user_id=b7c39d57989c43ec88dc094c22a2cdf2)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:12.499740 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3cc7d6-4212-4bff-9d49-d16f14b41488 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:12.501636 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 523/1063] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:07:12 2022] GET /identity/v3/users/b7c39d57989c43ec88dc094c22a2cdf2 => generated 285 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:12.508003 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad3ad9d-b25a-40aa-a2a7-3e5b3077df84 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:12.512005 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:12.587235 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3cc7d6-4212-4bff-9d49-d16f14b41488 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:12.616027 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad3ad9d-b25a-40aa-a2a7-3e5b3077df84 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:12.657960 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a34af4a-143c-4aae-9f21-0cc2a881488b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:12.658742 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34af4a-143c-4aae-9f21-0cc2a881488b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:12.659019 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34af4a-143c-4aae-9f21-0cc2a881488b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:12.659288 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34af4a-143c-4aae-9f21-0cc2a881488b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:12.701109 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3cc7d6-4212-4bff-9d49-d16f14b41488 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:12.725710 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a34af4a-143c-4aae-9f21-0cc2a881488b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:12.727206 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a34af4a-143c-4aae-9f21-0cc2a881488b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:12.729784 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:12.734554 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c023f5-d958-4886-9a9d-304e83ab8ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:12.735248 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c023f5-d958-4886-9a9d-304e83ab8ee3 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:12.735532 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c023f5-d958-4886-9a9d-304e83ab8ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:12.735801 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c023f5-d958-4886-9a9d-304e83ab8ee3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:12.746122 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c023f5-d958-4886-9a9d-304e83ab8ee3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:07:12.750866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-45c023f5-d958-4886-9a9d-304e83ab8ee3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 05:07:12.750866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:07:12.750866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:07:12.750866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:07:12.750866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:07:12.750866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:07:12.750866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:07:12.750866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:07:12.750866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:07:12.750866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:07:12.750866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:07:12.750866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:07:12.750866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:07:12.750866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:07:12.750866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:07:12.750866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:07:12.751847 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:07:12.751847 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:07:12.751847 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:07:12.751847 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:07:12.751847 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:07:12.751847 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:07:12.751847 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:07:12.751847 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:07:12.751847 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:07:12.751847 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:07:12.751847 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:07:12.751847 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:07:12.751847 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:07:12.751847 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:07:12.751847 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:07:12.751847 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:07:12.752725 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:07:12.752725 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:07:12.752725 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:07:12.752725 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:07:12.752725 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:07:12.752725 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:07:12.752725 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:07:12.752725 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:07:12.752725 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:07:12.752725 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:07:12.752725 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 05:07:12.752725 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:07:12.752725 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 524/1064] 10.222.0.36 () {56 vars in 1120 bytes} [Wed Aug 31 05:07:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:07:12.757860 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c023f5-d958-4886-9a9d-304e83ab8ee3 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:12.772659 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:12.833410 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 541/1065] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:07:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 05:07:12.834613 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a34af4a-143c-4aae-9f21-0cc2a881488b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:12.858262 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c023f5-d958-4886-9a9d-304e83ab8ee3 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:12.892258 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5697b7-5265-45b7-b2ec-5ee3073e34e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:12.892853 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5697b7-5265-45b7-b2ec-5ee3073e34e5 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:12.893127 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5697b7-5265-45b7-b2ec-5ee3073e34e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:12.893395 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5697b7-5265-45b7-b2ec-5ee3073e34e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:12.894087 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5697b7-5265-45b7-b2ec-5ee3073e34e5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:12.894387 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5697b7-5265-45b7-b2ec-5ee3073e34e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:12.895803 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5697b7-5265-45b7-b2ec-5ee3073e34e5 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:12.903480 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 525/1066] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Aug 31 05:07:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:07:12.909261 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5697b7-5265-45b7-b2ec-5ee3073e34e5 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:12.932696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a34af4a-143c-4aae-9f21-0cc2a881488b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:12.935024 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:12.967859 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe75939-2a88-423a-b4a0-b0a6aa2518da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:12.968417 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe75939-2a88-423a-b4a0-b0a6aa2518da None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:12.968682 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe75939-2a88-423a-b4a0-b0a6aa2518da None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:12.968952 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe75939-2a88-423a-b4a0-b0a6aa2518da None admin] PATH_INFO: `/v3/projects/6647c4a49c894179b981c3f9e904b7c4` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:12.970115 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe75939-2a88-423a-b4a0-b0a6aa2518da None admin] RBAC: Authorizing `identity:get_project(project_id=6647c4a49c894179b981c3f9e904b7c4)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:12.971556 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe75939-2a88-423a-b4a0-b0a6aa2518da None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:12.973223 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 542/1067] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:07:12 2022] GET /identity/v3/projects/6647c4a49c894179b981c3f9e904b7c4 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:12.980115 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe75939-2a88-423a-b4a0-b0a6aa2518da None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:12.982737 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:13.012624 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5697b7-5265-45b7-b2ec-5ee3073e34e5 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:13.047577 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f02227-a02a-438f-9964-5ee6a6b66abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:13.048210 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f02227-a02a-438f-9964-5ee6a6b66abf None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:13.048487 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f02227-a02a-438f-9964-5ee6a6b66abf None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:13.048752 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f02227-a02a-438f-9964-5ee6a6b66abf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:13.059317 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f02227-a02a-438f-9964-5ee6a6b66abf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:13.060774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f02227-a02a-438f-9964-5ee6a6b66abf None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:13.074683 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-96f02227-a02a-438f-9964-5ee6a6b66abf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:07:13.074683 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:07:13.074683 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:07:13.074683 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:07:13.074683 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:07:13.074683 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:07:13.074683 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:07:13.074683 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:07:13.074683 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:07:13.074683 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:07:13.074683 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:07:13.074683 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:07:13.074683 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:07:13.074683 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:07:13.074683 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:07:13.074683 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:07:13.075604 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:07:13.075604 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:07:13.075604 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:07:13.075604 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:07:13.075604 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:07:13.075604 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:07:13.075604 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:07:13.075604 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:07:13.075604 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:07:13.075604 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:07:13.075604 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:07:13.075604 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:07:13.075604 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:07:13.075604 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:07:13.075604 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:07:13.075604 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:07:13.075604 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:07:13.076497 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:07:13.076497 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:07:13.076497 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:07:13.076497 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 526/1068] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:07:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:07:13.081618 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f02227-a02a-438f-9964-5ee6a6b66abf None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:13.083863 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe75939-2a88-423a-b4a0-b0a6aa2518da None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:13.119702 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db448b-294e-488a-b0f4-2932bc99d458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:13.120311 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db448b-294e-488a-b0f4-2932bc99d458 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:13.120585 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db448b-294e-488a-b0f4-2932bc99d458 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:13.120885 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db448b-294e-488a-b0f4-2932bc99d458 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:13.121595 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6db448b-294e-488a-b0f4-2932bc99d458 None admin] RBAC: Adding query filter params (role.id=c3d5589dd2864be9a0f26f5802d874cd, scope.project.id=6647c4a49c894179b981c3f9e904b7c4, user.id=b7c39d57989c43ec88dc094c22a2cdf2) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:13.121897 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6db448b-294e-488a-b0f4-2932bc99d458 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:13.123356 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6db448b-294e-488a-b0f4-2932bc99d458 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:13.128670 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 543/1069] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:07:12 2022] GET /identity/v3/role_assignments?user.id=b7c39d57989c43ec88dc094c22a2cdf2&scope.project.id=6647c4a49c894179b981c3f9e904b7c4&role.id=c3d5589dd2864be9a0f26f5802d874cd => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:13.187342 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f02227-a02a-438f-9964-5ee6a6b66abf None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:13.215213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:13.225058 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02b0b4c-5261-49c8-b89a-ca5f6200a3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:13.225579 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b0b4c-5261-49c8-b89a-ca5f6200a3a6 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:13.225857 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b0b4c-5261-49c8-b89a-ca5f6200a3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:13.226153 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b0b4c-5261-49c8-b89a-ca5f6200a3a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:13.226796 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02b0b4c-5261-49c8-b89a-ca5f6200a3a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:13.227095 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02b0b4c-5261-49c8-b89a-ca5f6200a3a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:13.228495 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02b0b4c-5261-49c8-b89a-ca5f6200a3a6 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:13.241937 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 527/1070] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:07:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:07:13.248300 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db448b-294e-488a-b0f4-2932bc99d458 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:13.287192 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:13.351050 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db448b-294e-488a-b0f4-2932bc99d458 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:13.388829 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c68a04-b213-4e63-bf4f-bedaa240528e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:13.389445 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c68a04-b213-4e63-bf4f-bedaa240528e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:13.389717 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c68a04-b213-4e63-bf4f-bedaa240528e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:13.389977 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c68a04-b213-4e63-bf4f-bedaa240528e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:13.397331 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c68a04-b213-4e63-bf4f-bedaa240528e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:13.398797 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c68a04-b213-4e63-bf4f-bedaa240528e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:13.410937 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-65c68a04-b213-4e63-bf4f-bedaa240528e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 05:07:13.410937 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:07:13.410937 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:07:13.410937 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:07:13.410937 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:07:13.410937 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:07:13.410937 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:07:13.410937 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:07:13.410937 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:07:13.410937 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:07:13.410937 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:07:13.410937 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:07:13.410937 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 05:07:13.410937 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 05:07:13.410937 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 05:07:13.410937 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 05:07:13.411960 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:07:13.411960 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:07:13.411960 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 05:07:13.411960 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:07:13.411960 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 05:07:13.411960 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:07:13.411960 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:07:13.411960 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:07:13.411960 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:07:13.411960 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:07:13.411960 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:07:13.411960 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:07:13.411960 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:07:13.411960 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:07:13.411960 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:07:13.411960 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:07:13.411960 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:07:13.411960 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:07:13.413297 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:07:13.413297 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:07:13.413297 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 05:07:13.413297 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 05:07:13.413297 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 05:07:13.413297 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 05:07:13.413297 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 05:07:13.413297 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 05:07:13.413297 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 05:07:13.413297 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:07:13.413297 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 544/1071] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Aug 31 05:07:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:07:13.418074 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02b0b4c-5261-49c8-b89a-ca5f6200a3a6 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:13.525803 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02b0b4c-5261-49c8-b89a-ca5f6200a3a6 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:13.561199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d833373-0038-407b-b04b-11b8369bc7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:13.561807 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d833373-0038-407b-b04b-11b8369bc7ad None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:13.562113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d833373-0038-407b-b04b-11b8369bc7ad None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:13.562381 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d833373-0038-407b-b04b-11b8369bc7ad None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:13.563108 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d833373-0038-407b-b04b-11b8369bc7ad None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:13.563409 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d833373-0038-407b-b04b-11b8369bc7ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:13.564857 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d833373-0038-407b-b04b-11b8369bc7ad None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:13.581940 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 528/1072] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Aug 31 05:07:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:13.587603 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c68a04-b213-4e63-bf4f-bedaa240528e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:13.688341 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:13.695251 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c68a04-b213-4e63-bf4f-bedaa240528e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:13.729771 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce5bb1-48f4-47df-ba5e-14ae58645f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:13.730455 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce5bb1-48f4-47df-ba5e-14ae58645f87 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:13.730726 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce5bb1-48f4-47df-ba5e-14ae58645f87 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:13.730988 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce5bb1-48f4-47df-ba5e-14ae58645f87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:13.741481 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce5bb1-48f4-47df-ba5e-14ae58645f87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:13.743273 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce5bb1-48f4-47df-ba5e-14ae58645f87 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:13.757111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-0fce5bb1-48f4-47df-ba5e-14ae58645f87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:07:13.757111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:07:13.757111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:07:13.757111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:07:13.757111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:07:13.757111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:07:13.757111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:07:13.757111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:07:13.757111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:07:13.757111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:07:13.757111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:07:13.757111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:07:13.757111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:07:13.757111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:07:13.757111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:07:13.757111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:07:13.758217 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:07:13.758217 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:07:13.758217 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:07:13.758217 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:07:13.758217 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:07:13.758217 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:07:13.758217 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:07:13.758217 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:07:13.758217 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:07:13.758217 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:07:13.758217 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:07:13.758217 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:07:13.758217 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:07:13.758217 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:07:13.758217 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:07:13.758217 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:07:13.758217 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:07:13.759762 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:07:13.759762 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:07:13.759762 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:07:13.759762 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 545/1073] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:07:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:07:13.764332 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d833373-0038-407b-b04b-11b8369bc7ad None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:13.808533 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:13.861786 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d833373-0038-407b-b04b-11b8369bc7ad None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:13.897596 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f8eb2e-4638-48c3-a052-5569240c1031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:13.898217 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f8eb2e-4638-48c3-a052-5569240c1031 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:13.898489 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f8eb2e-4638-48c3-a052-5569240c1031 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:13.898751 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f8eb2e-4638-48c3-a052-5569240c1031 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:13.899382 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f8eb2e-4638-48c3-a052-5569240c1031 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:13.899683 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f8eb2e-4638-48c3-a052-5569240c1031 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:13.901047 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f8eb2e-4638-48c3-a052-5569240c1031 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:13.912355 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 529/1074] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:07:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:13.918132 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce5bb1-48f4-47df-ba5e-14ae58645f87 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:14.022179 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce5bb1-48f4-47df-ba5e-14ae58645f87 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:14.058987 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df5d2f9-ae98-414e-97a7-479a878916bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:14.059526 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df5d2f9-ae98-414e-97a7-479a878916bc None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:14.059796 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df5d2f9-ae98-414e-97a7-479a878916bc None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:14.060052 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df5d2f9-ae98-414e-97a7-479a878916bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:14.068381 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df5d2f9-ae98-414e-97a7-479a878916bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:14.069815 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df5d2f9-ae98-414e-97a7-479a878916bc None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:14.081284 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-8df5d2f9-ae98-414e-97a7-479a878916bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:07:14.081284 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:07:14.081284 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:07:14.081284 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:07:14.081284 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:07:14.081284 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:07:14.081284 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:07:14.081284 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:07:14.081284 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:07:14.081284 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:07:14.081284 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:07:14.081284 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:07:14.081284 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 05:07:14.081284 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 05:07:14.081284 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 05:07:14.082308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 05:07:14.082308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:07:14.082308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:07:14.082308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:07:14.082308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:07:14.082308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:07:14.082308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:07:14.082308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:07:14.082308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:07:14.082308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:07:14.082308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:07:14.082308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:07:14.082308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:07:14.082308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:07:14.082308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:07:14.082308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:07:14.082308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:07:14.083783 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 05:07:14.083783 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 05:07:14.083783 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 05:07:14.083783 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 05:07:14.083783 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 05:07:14.083783 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 05:07:14.083783 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:07:14.083783 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:07:14.083783 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 546/1075] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:07:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:07:14.088626 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f8eb2e-4638-48c3-a052-5569240c1031 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:14.193180 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f8eb2e-4638-48c3-a052-5569240c1031 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:14.195608 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:14.231388 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d36511-55ab-474f-a587-df37f4f24c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:14.231997 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d36511-55ab-474f-a587-df37f4f24c5a None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:14.232275 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d36511-55ab-474f-a587-df37f4f24c5a None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:14.232542 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d36511-55ab-474f-a587-df37f4f24c5a None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:14.233235 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d36511-55ab-474f-a587-df37f4f24c5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:14.233540 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d36511-55ab-474f-a587-df37f4f24c5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:14.235016 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d36511-55ab-474f-a587-df37f4f24c5a None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:14.247469 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 530/1076] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 05:07:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:14.253529 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df5d2f9-ae98-414e-97a7-479a878916bc None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:14.265357 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:14.354624 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df5d2f9-ae98-414e-97a7-479a878916bc None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:14.389036 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f4f8f1-d219-432b-9742-1cdcbb0ad19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:14.389670 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4f8f1-d219-432b-9742-1cdcbb0ad19c None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:14.389945 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4f8f1-d219-432b-9742-1cdcbb0ad19c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:14.390245 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4f8f1-d219-432b-9742-1cdcbb0ad19c None admin] PATH_INFO: `/v3/projects/e01ce1b1282e44a39d659b87f120995e/users/f27665b1574a4db7b7a9771f7d775826/roles/f7c0c9eac46c42179e586dc4c1726356` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:14.392414 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f4f8f1-d219-432b-9742-1cdcbb0ad19c None admin] RBAC: Authorizing `identity:create_grant(project_id=e01ce1b1282e44a39d659b87f120995e, user_id=f27665b1574a4db7b7a9771f7d775826, role_id=f7c0c9eac46c42179e586dc4c1726356)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:14.393849 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f4f8f1-d219-432b-9742-1cdcbb0ad19c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:14.408061 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 547/1077] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 05:07:14 2022] PUT /identity/v3/projects/e01ce1b1282e44a39d659b87f120995e/users/f27665b1574a4db7b7a9771f7d775826/roles/f7c0c9eac46c42179e586dc4c1726356 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:07:14.993012 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64f357-e29d-4909-ab26-33c96e787f2f None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:14.993527 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64f357-e29d-4909-ab26-33c96e787f2f None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:14.993527 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64f357-e29d-4909-ab26-33c96e787f2f None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:14.994940 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 531/1078] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:07:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:07:15.000252 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe81090b-be07-4bdb-9d95-b4c2df54672c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:15.000536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe81090b-be07-4bdb-9d95-b4c2df54672c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:15.000812 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe81090b-be07-4bdb-9d95-b4c2df54672c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:15.019878 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-fe81090b-be07-4bdb-9d95-b4c2df54672c None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:07:15.021039 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fe81090b-be07-4bdb-9d95-b4c2df54672c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:07:15.083883 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:15.093823 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 548/1079] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:07:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:07:15.356487 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64f357-e29d-4909-ab26-33c96e787f2f None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:15.467792 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64f357-e29d-4909-ab26-33c96e787f2f None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:15.502441 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1789c197-9f68-4f63-a945-56f356699bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:15.503060 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789c197-9f68-4f63-a945-56f356699bf3 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:15.503339 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789c197-9f68-4f63-a945-56f356699bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:15.503603 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789c197-9f68-4f63-a945-56f356699bf3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:15.513572 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1789c197-9f68-4f63-a945-56f356699bf3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:07:15.518337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-1789c197-9f68-4f63-a945-56f356699bf3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:07:15.518337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:07:15.518337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:07:15.518337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:07:15.518337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:07:15.518337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:07:15.518337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:07:15.518337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:07:15.518337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:07:15.518337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:07:15.518337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:07:15.518337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:07:15.518337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:07:15.518337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:07:15.518337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:07:15.518337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:07:15.519310 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:07:15.519310 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:07:15.519310 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:07:15.519310 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:07:15.519310 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:07:15.519310 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:07:15.519310 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:07:15.519310 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:07:15.519310 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:07:15.519310 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:07:15.519310 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:07:15.519310 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:07:15.519310 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:07:15.519310 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:07:15.519310 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:07:15.519310 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:07:15.520160 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:07:15.520160 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:07:15.520160 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:07:15.520160 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:07:15.520160 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:07:15.520160 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:07:15.520160 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:07:15.520160 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:07:15.520160 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:07:15.520160 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:07:15.520160 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:07:15.520160 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:07:15.520160 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 532/1080] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 05:07:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:07:15.525596 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe81090b-be07-4bdb-9d95-b4c2df54672c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:15.627319 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe81090b-be07-4bdb-9d95-b4c2df54672c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:15.662443 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def26971-531a-4e56-abfc-764a6b1deec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:15.663040 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def26971-531a-4e56-abfc-764a6b1deec5 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:15.663311 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def26971-531a-4e56-abfc-764a6b1deec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:15.663571 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def26971-531a-4e56-abfc-764a6b1deec5 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:15.664229 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def26971-531a-4e56-abfc-764a6b1deec5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:15.664526 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def26971-531a-4e56-abfc-764a6b1deec5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:15.665877 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def26971-531a-4e56-abfc-764a6b1deec5 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:15.674371 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 549/1081] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:07:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:15.680134 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1789c197-9f68-4f63-a945-56f356699bf3 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:15.706279 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:15.783389 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1789c197-9f68-4f63-a945-56f356699bf3 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:15.818032 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47981040-30b8-45d1-a2d2-50609d7ea4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:15.818657 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47981040-30b8-45d1-a2d2-50609d7ea4df None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:15.818928 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47981040-30b8-45d1-a2d2-50609d7ea4df None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:15.819192 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47981040-30b8-45d1-a2d2-50609d7ea4df None admin] PATH_INFO: `/v3/users/b7c39d57989c43ec88dc094c22a2cdf2` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:15.820338 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47981040-30b8-45d1-a2d2-50609d7ea4df None admin] RBAC: Authorizing `identity:get_user(user_id=b7c39d57989c43ec88dc094c22a2cdf2)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:15.821773 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47981040-30b8-45d1-a2d2-50609d7ea4df None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:15.823392 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 533/1082] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:07:15 2022] GET /identity/v3/users/b7c39d57989c43ec88dc094c22a2cdf2 => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:15.829338 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def26971-531a-4e56-abfc-764a6b1deec5 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:15.891468 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:15.934890 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def26971-531a-4e56-abfc-764a6b1deec5 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:15.967156 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dd1956-a809-4e22-8a09-6189fe1cb66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:15.967773 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd1956-a809-4e22-8a09-6189fe1cb66b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:15.968043 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd1956-a809-4e22-8a09-6189fe1cb66b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:15.968304 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd1956-a809-4e22-8a09-6189fe1cb66b None admin] PATH_INFO: `/v3/projects/6647c4a49c894179b981c3f9e904b7c4` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:15.969383 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dd1956-a809-4e22-8a09-6189fe1cb66b None admin] RBAC: Authorizing `identity:get_project(project_id=6647c4a49c894179b981c3f9e904b7c4)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:15.970838 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dd1956-a809-4e22-8a09-6189fe1cb66b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:15.972464 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 550/1083] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:07:15 2022] GET /identity/v3/projects/6647c4a49c894179b981c3f9e904b7c4 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:15.978539 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47981040-30b8-45d1-a2d2-50609d7ea4df None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:16.071094 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:16.081659 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47981040-30b8-45d1-a2d2-50609d7ea4df None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:16.116507 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e9ae5f-dd1a-42f0-82e5-3deadaa975c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:16.117165 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e9ae5f-dd1a-42f0-82e5-3deadaa975c9 None admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:16.117749 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e9ae5f-dd1a-42f0-82e5-3deadaa975c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:16.118067 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e9ae5f-dd1a-42f0-82e5-3deadaa975c9 None admin] PATH_INFO: `/v3/projects/6647c4a49c894179b981c3f9e904b7c4/users/b7c39d57989c43ec88dc094c22a2cdf2/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:16.119956 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e9ae5f-dd1a-42f0-82e5-3deadaa975c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=6647c4a49c894179b981c3f9e904b7c4, user_id=b7c39d57989c43ec88dc094c22a2cdf2, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:16.121445 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e9ae5f-dd1a-42f0-82e5-3deadaa975c9 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:16.135497 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 534/1084] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 05:07:15 2022] PUT /identity/v3/projects/6647c4a49c894179b981c3f9e904b7c4/users/b7c39d57989c43ec88dc094c22a2cdf2/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:07:16.272928 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbcf835-eae5-4d91-b658-adba91710da1 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:16.273285 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbcf835-eae5-4d91-b658-adba91710da1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:16.273426 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbcf835-eae5-4d91-b658-adba91710da1 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:16.274810 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 551/1085] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:07:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:07:16.280447 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b329752-4ce4-4c87-b422-d1dd892ad423 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:16.280726 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b329752-4ce4-4c87-b422-d1dd892ad423 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:16.280992 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b329752-4ce4-4c87-b422-d1dd892ad423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:16.299692 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-9b329752-4ce4-4c87-b422-d1dd892ad423 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:07:16.300868 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-9b329752-4ce4-4c87-b422-d1dd892ad423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:07:16.369449 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 535/1086] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:07:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:07:16.631454 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbcf835-eae5-4d91-b658-adba91710da1 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:16.737153 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbcf835-eae5-4d91-b658-adba91710da1 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:16.762675 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:16.771004 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a17f1-5b08-44db-9cfe-21a9a274b386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:16.771661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a17f1-5b08-44db-9cfe-21a9a274b386 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:16.771934 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a17f1-5b08-44db-9cfe-21a9a274b386 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:16.772202 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a17f1-5b08-44db-9cfe-21a9a274b386 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:16.783448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464a17f1-5b08-44db-9cfe-21a9a274b386 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:07:16.788136 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-464a17f1-5b08-44db-9cfe-21a9a274b386 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 05:07:16.788136 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:07:16.788136 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:07:16.788136 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:07:16.788136 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:07:16.788136 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:07:16.788136 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:07:16.788136 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:07:16.788136 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:07:16.788136 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:07:16.788136 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:07:16.788136 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:07:16.788136 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:07:16.788136 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:07:16.788136 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:07:16.788136 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:07:16.789111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:07:16.789111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:07:16.789111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:07:16.789111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:07:16.789111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:07:16.789111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:07:16.789111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:07:16.789111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:07:16.789111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:07:16.789111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:07:16.789111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:07:16.789111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:07:16.789111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:07:16.789111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:07:16.789111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:07:16.789111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:07:16.789946 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:07:16.789946 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:07:16.789946 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:07:16.789946 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:07:16.789946 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:07:16.789946 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:07:16.789946 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:07:16.789946 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:07:16.789946 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:07:16.789946 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:07:16.789946 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 05:07:16.789946 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:07:16.789946 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 552/1087] 10.222.0.36 () {56 vars in 1120 bytes} [Wed Aug 31 05:07:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:07:16.795685 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b329752-4ce4-4c87-b422-d1dd892ad423 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:16.881440 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:16.900779 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b329752-4ce4-4c87-b422-d1dd892ad423 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:16.934645 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd568c2-cab6-4cda-aa68-e8d60c9f5731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:16.935247 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd568c2-cab6-4cda-aa68-e8d60c9f5731 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:16.935523 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd568c2-cab6-4cda-aa68-e8d60c9f5731 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:16.935788 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd568c2-cab6-4cda-aa68-e8d60c9f5731 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:16.936450 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd568c2-cab6-4cda-aa68-e8d60c9f5731 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:16.936748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd568c2-cab6-4cda-aa68-e8d60c9f5731 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:16.938140 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd568c2-cab6-4cda-aa68-e8d60c9f5731 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:16.946104 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 536/1088] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Aug 31 05:07:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:16.951828 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a17f1-5b08-44db-9cfe-21a9a274b386 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:17.054932 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a17f1-5b08-44db-9cfe-21a9a274b386 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:17.090615 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27efeb52-c417-4273-9ba4-32a11bbc227e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:17.091239 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27efeb52-c417-4273-9ba4-32a11bbc227e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:17.091512 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27efeb52-c417-4273-9ba4-32a11bbc227e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:17.091775 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27efeb52-c417-4273-9ba4-32a11bbc227e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:17.102622 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27efeb52-c417-4273-9ba4-32a11bbc227e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:17.104074 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27efeb52-c417-4273-9ba4-32a11bbc227e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:17.118628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-27efeb52-c417-4273-9ba4-32a11bbc227e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:07:17.118628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:07:17.118628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:07:17.118628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:07:17.118628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:07:17.118628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:07:17.118628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:07:17.118628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:07:17.118628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:07:17.118628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:07:17.118628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:07:17.118628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:07:17.118628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:07:17.118628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:07:17.118628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:07:17.118628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:07:17.119712 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:07:17.119712 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:07:17.119712 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:07:17.119712 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:07:17.119712 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:07:17.119712 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:07:17.119712 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:07:17.119712 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:07:17.119712 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:07:17.119712 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:07:17.119712 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:07:17.119712 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:07:17.119712 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:07:17.119712 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:07:17.119712 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:07:17.119712 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:07:17.119712 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:07:17.120813 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:07:17.120813 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:07:17.120813 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:07:17.120813 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 553/1089] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:07:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:07:17.125846 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd568c2-cab6-4cda-aa68-e8d60c9f5731 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:17.235673 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd568c2-cab6-4cda-aa68-e8d60c9f5731 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:17.270048 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bfcf4f-5ccc-48f7-831a-a70fc42460ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:17.270929 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bfcf4f-5ccc-48f7-831a-a70fc42460ce None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:17.271204 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bfcf4f-5ccc-48f7-831a-a70fc42460ce None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:17.271469 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bfcf4f-5ccc-48f7-831a-a70fc42460ce None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:17.272098 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bfcf4f-5ccc-48f7-831a-a70fc42460ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:17.272392 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bfcf4f-5ccc-48f7-831a-a70fc42460ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:17.273755 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bfcf4f-5ccc-48f7-831a-a70fc42460ce None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:17.285573 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 537/1090] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:07:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:17.291334 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27efeb52-c417-4273-9ba4-32a11bbc227e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:17.334827 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:17.394998 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27efeb52-c417-4273-9ba4-32a11bbc227e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:17.428928 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678cb89d-85a6-4c54-a6da-1b97ec88ab87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:17.429542 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678cb89d-85a6-4c54-a6da-1b97ec88ab87 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:17.429816 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678cb89d-85a6-4c54-a6da-1b97ec88ab87 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:17.430118 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678cb89d-85a6-4c54-a6da-1b97ec88ab87 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:17.437515 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678cb89d-85a6-4c54-a6da-1b97ec88ab87 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:17.438989 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678cb89d-85a6-4c54-a6da-1b97ec88ab87 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:17.449872 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-678cb89d-85a6-4c54-a6da-1b97ec88ab87 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 05:07:17.449872 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:07:17.449872 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:07:17.449872 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:07:17.449872 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:07:17.449872 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:07:17.449872 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:07:17.449872 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:07:17.449872 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:07:17.449872 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:07:17.449872 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:07:17.449872 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:07:17.449872 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 05:07:17.449872 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 05:07:17.449872 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 05:07:17.449872 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 05:07:17.450888 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:07:17.450888 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:07:17.450888 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 05:07:17.450888 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:07:17.450888 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 05:07:17.450888 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:07:17.450888 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:07:17.450888 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:07:17.450888 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:07:17.450888 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:07:17.450888 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:07:17.450888 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:07:17.450888 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:07:17.450888 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:07:17.450888 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:07:17.450888 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:07:17.450888 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:07:17.450888 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:07:17.452203 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:07:17.452203 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:07:17.452203 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 05:07:17.452203 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 05:07:17.452203 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 05:07:17.452203 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 05:07:17.452203 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 05:07:17.452203 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 05:07:17.452203 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 05:07:17.452203 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:07:17.452203 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 554/1091] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Aug 31 05:07:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:07:17.456901 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bfcf4f-5ccc-48f7-831a-a70fc42460ce None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:17.461053 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:17.559287 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bfcf4f-5ccc-48f7-831a-a70fc42460ce None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:17.594444 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64885d7a-4cee-4e9d-a0b6-c4e23340b27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:17.595039 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64885d7a-4cee-4e9d-a0b6-c4e23340b27f None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:17.595319 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64885d7a-4cee-4e9d-a0b6-c4e23340b27f None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:17.595579 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64885d7a-4cee-4e9d-a0b6-c4e23340b27f None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:17.596296 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64885d7a-4cee-4e9d-a0b6-c4e23340b27f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:17.596598 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64885d7a-4cee-4e9d-a0b6-c4e23340b27f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:17.598047 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64885d7a-4cee-4e9d-a0b6-c4e23340b27f None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:17.615383 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 538/1092] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Aug 31 05:07:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:17.621566 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678cb89d-85a6-4c54-a6da-1b97ec88ab87 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:17.721677 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:17.725561 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678cb89d-85a6-4c54-a6da-1b97ec88ab87 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:17.760647 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd1d64e-560a-403e-8411-944cbafab678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:17.761267 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd1d64e-560a-403e-8411-944cbafab678 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:17.761545 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd1d64e-560a-403e-8411-944cbafab678 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:17.761812 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd1d64e-560a-403e-8411-944cbafab678 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:17.832868 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd1d64e-560a-403e-8411-944cbafab678 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:17.834283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd1d64e-560a-403e-8411-944cbafab678 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:17.930630 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:17.935102 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 555/1093] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:07:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 05:07:17.940430 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64885d7a-4cee-4e9d-a0b6-c4e23340b27f None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:17.976262 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eaab0d-55a2-4bc7-a2d9-0c7ee787fdb5 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:17.976638 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eaab0d-55a2-4bc7-a2d9-0c7ee787fdb5 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:17.976900 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eaab0d-55a2-4bc7-a2d9-0c7ee787fdb5 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:17.978584 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 556/1094] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:07:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 05:07:17.983469 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7cad6-89ca-4d60-ae88-f54d328a4ebb None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:17.983750 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7cad6-89ca-4d60-ae88-f54d328a4ebb None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:17.984017 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7cad6-89ca-4d60-ae88-f54d328a4ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:18.001778 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-8ee7cad6-89ca-4d60-ae88-f54d328a4ebb None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:07:18.002310 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:18.003166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8ee7cad6-89ca-4d60-ae88-f54d328a4ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:07:18.041822 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64885d7a-4cee-4e9d-a0b6-c4e23340b27f None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:18.071305 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 557/1095] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:07:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 05:07:18.080202 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecf4f11-0aa6-407a-b939-4cde673fd186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:18.080829 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf4f11-0aa6-407a-b939-4cde673fd186 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:18.081099 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf4f11-0aa6-407a-b939-4cde673fd186 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:18.081361 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf4f11-0aa6-407a-b939-4cde673fd186 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:18.091682 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecf4f11-0aa6-407a-b939-4cde673fd186 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:18.093066 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecf4f11-0aa6-407a-b939-4cde673fd186 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:18.107570 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-aecf4f11-0aa6-407a-b939-4cde673fd186 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:07:18.107570 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:07:18.107570 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:07:18.107570 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:07:18.107570 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:07:18.107570 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:07:18.107570 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:07:18.107570 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:07:18.107570 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:07:18.107570 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:07:18.107570 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:07:18.107570 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:07:18.107570 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:07:18.107570 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:07:18.107570 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:07:18.107570 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:07:18.108560 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:07:18.108560 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:07:18.108560 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:07:18.108560 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:07:18.108560 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:07:18.108560 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:07:18.108560 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:07:18.108560 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:07:18.108560 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:07:18.108560 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:07:18.108560 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:07:18.108560 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:07:18.108560 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:07:18.108560 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:07:18.108560 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:07:18.108560 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:07:18.108560 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:07:18.109478 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:07:18.109478 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:07:18.109478 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:07:18.109478 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 539/1096] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:07:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:07:18.114525 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee7cad6-89ca-4d60-ae88-f54d328a4ebb None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:18.214621 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee7cad6-89ca-4d60-ae88-f54d328a4ebb None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:18.244619 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:18.251631 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a88e6d-2eb6-4c70-a190-739f50ecfbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:18.252241 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a88e6d-2eb6-4c70-a190-739f50ecfbdb None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:18.252518 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a88e6d-2eb6-4c70-a190-739f50ecfbdb None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:18.252780 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a88e6d-2eb6-4c70-a190-739f50ecfbdb None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:18.253450 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a88e6d-2eb6-4c70-a190-739f50ecfbdb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:18.253743 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a88e6d-2eb6-4c70-a190-739f50ecfbdb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:18.255144 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a88e6d-2eb6-4c70-a190-739f50ecfbdb None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:18.266686 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 558/1097] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:07:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:18.272455 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecf4f11-0aa6-407a-b939-4cde673fd186 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:18.336140 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a88e6d-2eb6-4c70-a190-739f50ecfbdb None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:18.375008 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecf4f11-0aa6-407a-b939-4cde673fd186 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:18.388160 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:18.411130 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e65f71-eb11-46c8-9aeb-a2eb43cd7b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:18.411751 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e65f71-eb11-46c8-9aeb-a2eb43cd7b80 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:18.412022 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e65f71-eb11-46c8-9aeb-a2eb43cd7b80 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:18.412287 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e65f71-eb11-46c8-9aeb-a2eb43cd7b80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:18.418895 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e65f71-eb11-46c8-9aeb-a2eb43cd7b80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:18.420377 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e65f71-eb11-46c8-9aeb-a2eb43cd7b80 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:18.431288 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-44e65f71-eb11-46c8-9aeb-a2eb43cd7b80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:07:18.431288 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:07:18.431288 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:07:18.431288 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:07:18.431288 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:07:18.431288 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:07:18.431288 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:07:18.431288 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:07:18.431288 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:07:18.431288 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:07:18.431288 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:07:18.431288 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:07:18.431288 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 05:07:18.431288 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 05:07:18.431288 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 05:07:18.432221 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 05:07:18.432221 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:07:18.432221 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:07:18.432221 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:07:18.432221 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:07:18.432221 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:07:18.432221 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:07:18.432221 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:07:18.432221 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:07:18.432221 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:07:18.432221 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:07:18.432221 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:07:18.432221 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:07:18.432221 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:07:18.432221 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:07:18.432221 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:07:18.432221 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:07:18.433210 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 05:07:18.433210 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 05:07:18.433210 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 05:07:18.433210 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 05:07:18.433210 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 05:07:18.433210 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 05:07:18.433210 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:07:18.433210 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:07:18.433210 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 540/1098] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:07:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:07:18.438282 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e65f71-eb11-46c8-9aeb-a2eb43cd7b80 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:18.440390 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a88e6d-2eb6-4c70-a190-739f50ecfbdb None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:18.472085 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d411364-3914-4147-862e-014b30933bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:18.472703 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d411364-3914-4147-862e-014b30933bbd None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:18.472975 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d411364-3914-4147-862e-014b30933bbd None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:18.473240 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d411364-3914-4147-862e-014b30933bbd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:18.483371 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d411364-3914-4147-862e-014b30933bbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:07:18.488098 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-7d411364-3914-4147-862e-014b30933bbd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:07:18.488098 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:07:18.488098 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:07:18.488098 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:07:18.488098 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:07:18.488098 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:07:18.488098 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:07:18.488098 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:07:18.488098 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:07:18.488098 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:07:18.488098 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:07:18.488098 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:07:18.488098 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:07:18.488098 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:07:18.488098 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:07:18.488098 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:07:18.489047 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:07:18.489047 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:07:18.489047 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:07:18.489047 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:07:18.489047 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:07:18.489047 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:07:18.489047 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:07:18.489047 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:07:18.489047 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:07:18.489047 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:07:18.489047 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:07:18.489047 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:07:18.489047 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:07:18.489047 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:07:18.489047 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:07:18.489047 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:07:18.489902 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:07:18.489902 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:07:18.489902 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:07:18.489902 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:07:18.489902 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:07:18.489902 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:07:18.489902 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:07:18.489902 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:07:18.489902 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:07:18.489902 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:07:18.489902 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:07:18.489902 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:07:18.489902 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 559/1099] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 05:07:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:07:18.495502 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d411364-3914-4147-862e-014b30933bbd None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:18.535304 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e65f71-eb11-46c8-9aeb-a2eb43cd7b80 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:18.544544 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:18.567723 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7458e6-b4e8-46df-9408-3008e4e3d3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:18.568325 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7458e6-b4e8-46df-9408-3008e4e3d3dd None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:18.568599 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7458e6-b4e8-46df-9408-3008e4e3d3dd None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:18.568866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7458e6-b4e8-46df-9408-3008e4e3d3dd None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:18.569556 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7458e6-b4e8-46df-9408-3008e4e3d3dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:18.569854 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7458e6-b4e8-46df-9408-3008e4e3d3dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:18.571335 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7458e6-b4e8-46df-9408-3008e4e3d3dd None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:18.583427 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 541/1100] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 05:07:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:07:18.589461 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7458e6-b4e8-46df-9408-3008e4e3d3dd None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:18.594263 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d411364-3914-4147-862e-014b30933bbd None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:18.624374 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:18.627892 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c9cce-4c1a-4c5d-8b71-68d06d0126a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:18.628428 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c9cce-4c1a-4c5d-8b71-68d06d0126a2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:18.628702 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c9cce-4c1a-4c5d-8b71-68d06d0126a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:18.628964 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c9cce-4c1a-4c5d-8b71-68d06d0126a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:18.629625 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39c9cce-4c1a-4c5d-8b71-68d06d0126a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:18.629919 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39c9cce-4c1a-4c5d-8b71-68d06d0126a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:18.631300 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39c9cce-4c1a-4c5d-8b71-68d06d0126a2 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:18.640330 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 560/1101] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:07:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:07:18.645873 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c9cce-4c1a-4c5d-8b71-68d06d0126a2 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:18.694192 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7458e6-b4e8-46df-9408-3008e4e3d3dd None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:18.728393 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c6a1a-6dff-4a84-9a68-55a537aa723b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:18.728994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c6a1a-6dff-4a84-9a68-55a537aa723b None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:18.729267 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c6a1a-6dff-4a84-9a68-55a537aa723b None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:18.729534 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c6a1a-6dff-4a84-9a68-55a537aa723b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:18.730311 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39c6a1a-6dff-4a84-9a68-55a537aa723b None admin] RBAC: Adding query filter params (role.id=f7c0c9eac46c42179e586dc4c1726356, scope.project.id=e01ce1b1282e44a39d659b87f120995e, user.id=f27665b1574a4db7b7a9771f7d775826) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:18.730612 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39c6a1a-6dff-4a84-9a68-55a537aa723b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:18.732046 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39c6a1a-6dff-4a84-9a68-55a537aa723b None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:18.737881 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 542/1102] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:07:18 2022] GET /identity/v3/role_assignments?user.id=f27665b1574a4db7b7a9771f7d775826&scope.project.id=e01ce1b1282e44a39d659b87f120995e&role.id=f7c0c9eac46c42179e586dc4c1726356 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:18.743192 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c9cce-4c1a-4c5d-8b71-68d06d0126a2 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:18.764360 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:18.789998 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e890cfc-0825-4bf5-b0ff-bb107314d3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:18.790682 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e890cfc-0825-4bf5-b0ff-bb107314d3b5 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:18.790953 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e890cfc-0825-4bf5-b0ff-bb107314d3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:18.791219 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e890cfc-0825-4bf5-b0ff-bb107314d3b5 None admin] PATH_INFO: `/v3/users/b7c39d57989c43ec88dc094c22a2cdf2` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:18.792535 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e890cfc-0825-4bf5-b0ff-bb107314d3b5 None admin] RBAC: Authorizing `identity:get_user(user_id=b7c39d57989c43ec88dc094c22a2cdf2)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:18.794058 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e890cfc-0825-4bf5-b0ff-bb107314d3b5 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:18.795745 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 561/1103] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:07:18 2022] GET /identity/v3/users/b7c39d57989c43ec88dc094c22a2cdf2 => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:07:18.802329 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c6a1a-6dff-4a84-9a68-55a537aa723b None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:18.905521 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:18.910891 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c6a1a-6dff-4a84-9a68-55a537aa723b None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:18.945356 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f229f5-dd36-47d0-ba5b-64c6f3d1b4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:18.945959 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f229f5-dd36-47d0-ba5b-64c6f3d1b4b7 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:18.946258 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f229f5-dd36-47d0-ba5b-64c6f3d1b4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:18.946526 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f229f5-dd36-47d0-ba5b-64c6f3d1b4b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:19.011856 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f229f5-dd36-47d0-ba5b-64c6f3d1b4b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:19.013309 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f229f5-dd36-47d0-ba5b-64c6f3d1b4b7 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:19.106861 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:19.114224 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 543/1104] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:07:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 05:07:19.118079 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e890cfc-0825-4bf5-b0ff-bb107314d3b5 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:19.225301 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e890cfc-0825-4bf5-b0ff-bb107314d3b5 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:19.261158 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ee5bc4-e77c-4d85-80da-d926fb16e603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:19.261795 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee5bc4-e77c-4d85-80da-d926fb16e603 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:19.262101 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee5bc4-e77c-4d85-80da-d926fb16e603 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:19.262373 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee5bc4-e77c-4d85-80da-d926fb16e603 None admin] PATH_INFO: `/v3/projects/6647c4a49c894179b981c3f9e904b7c4` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:19.263517 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ee5bc4-e77c-4d85-80da-d926fb16e603 None admin] RBAC: Authorizing `identity:get_project(project_id=6647c4a49c894179b981c3f9e904b7c4)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:19.264960 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ee5bc4-e77c-4d85-80da-d926fb16e603 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:19.266980 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 562/1105] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:07:19 2022] GET /identity/v3/projects/6647c4a49c894179b981c3f9e904b7c4 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:07:19.273983 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f229f5-dd36-47d0-ba5b-64c6f3d1b4b7 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:19.374225 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f229f5-dd36-47d0-ba5b-64c6f3d1b4b7 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:19.407748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75777fa1-a318-4b87-8328-e0479f98665f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:19.408368 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75777fa1-a318-4b87-8328-e0479f98665f None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:19.408646 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75777fa1-a318-4b87-8328-e0479f98665f None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:19.408912 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75777fa1-a318-4b87-8328-e0479f98665f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:19.409638 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75777fa1-a318-4b87-8328-e0479f98665f None admin] RBAC: Adding query filter params (role.id=c3d5589dd2864be9a0f26f5802d874cd, scope.project.id=6647c4a49c894179b981c3f9e904b7c4, user.id=b7c39d57989c43ec88dc094c22a2cdf2) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:19.409942 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75777fa1-a318-4b87-8328-e0479f98665f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:19.411409 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75777fa1-a318-4b87-8328-e0479f98665f None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:19.417424 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 544/1106] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:07:19 2022] GET /identity/v3/role_assignments?user.id=b7c39d57989c43ec88dc094c22a2cdf2&scope.project.id=6647c4a49c894179b981c3f9e904b7c4&role.id=c3d5589dd2864be9a0f26f5802d874cd => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:21.312387 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322ff38-b844-4838-ad7d-be0da90c5430 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:21.313204 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322ff38-b844-4838-ad7d-be0da90c5430 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:21.313204 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322ff38-b844-4838-ad7d-be0da90c5430 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:21.315438 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 563/1107] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:07:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:07:21.323302 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e7232c-b4c5-40ad-b4b1-511cf1f5eeb1 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:21.323814 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e7232c-b4c5-40ad-b4b1-511cf1f5eeb1 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:21.323903 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e7232c-b4c5-40ad-b4b1-511cf1f5eeb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:21.342465 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-c9e7232c-b4c5-40ad-b4b1-511cf1f5eeb1 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:07:21.343631 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c9e7232c-b4c5-40ad-b4b1-511cf1f5eeb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:07:21.403153 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:21.416135 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 545/1108] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:07:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 05:07:21.681437 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322ff38-b844-4838-ad7d-be0da90c5430 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:21.749648 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:21.803858 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322ff38-b844-4838-ad7d-be0da90c5430 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:21.839265 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d720ece-f3c9-4619-89e1-8a7e3314c3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:21.839880 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d720ece-f3c9-4619-89e1-8a7e3314c3fa None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:21.840154 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d720ece-f3c9-4619-89e1-8a7e3314c3fa None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:21.840420 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d720ece-f3c9-4619-89e1-8a7e3314c3fa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:21.841637 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d720ece-f3c9-4619-89e1-8a7e3314c3fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:21.843091 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d720ece-f3c9-4619-89e1-8a7e3314c3fa None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:21.844653 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 564/1109] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:07:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:21.853813 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e7232c-b4c5-40ad-b4b1-511cf1f5eeb1 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:21.961515 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e7232c-b4c5-40ad-b4b1-511cf1f5eeb1 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:21.996905 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00b5be-8cea-4069-bbd5-ca66bec4e446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:21.997495 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00b5be-8cea-4069-bbd5-ca66bec4e446 None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:21.997764 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00b5be-8cea-4069-bbd5-ca66bec4e446 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:21.998032 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00b5be-8cea-4069-bbd5-ca66bec4e446 None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:21.998700 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a00b5be-8cea-4069-bbd5-ca66bec4e446 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:22.000128 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a00b5be-8cea-4069-bbd5-ca66bec4e446 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:22.021579 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 546/1110] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Aug 31 05:07:21 2022] POST /identity/v3/users => generated 286 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:07:23.949711 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982801b-919d-405e-89df-a89a43e49663 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:23.950611 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982801b-919d-405e-89df-a89a43e49663 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:23.950611 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982801b-919d-405e-89df-a89a43e49663 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:23.952714 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 565/1111] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:07:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:07:23.958956 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47005b7-f1a8-4383-bb6b-39707c3c981a None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:23.959671 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47005b7-f1a8-4383-bb6b-39707c3c981a None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:23.959816 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47005b7-f1a8-4383-bb6b-39707c3c981a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:23.981185 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-b47005b7-f1a8-4383-bb6b-39707c3c981a None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:07:23.982407 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b47005b7-f1a8-4383-bb6b-39707c3c981a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:07:24.032810 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:24.057851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 547/1112] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:07:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:07:24.321330 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2982801b-919d-405e-89df-a89a43e49663 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:24.437244 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2982801b-919d-405e-89df-a89a43e49663 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:24.454077 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:24.475388 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693bbd83-df28-4651-9d2d-5a12ad859bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:24.476014 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693bbd83-df28-4651-9d2d-5a12ad859bd7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:24.476287 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693bbd83-df28-4651-9d2d-5a12ad859bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:24.476550 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693bbd83-df28-4651-9d2d-5a12ad859bd7 None admin] PATH_INFO: `/v3/roles/a74bf01dd66f49f5988a197da269c822` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:24.478034 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693bbd83-df28-4651-9d2d-5a12ad859bd7 None admin] RBAC: Authorizing `identity:get_role(role_id=a74bf01dd66f49f5988a197da269c822)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:24.479455 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693bbd83-df28-4651-9d2d-5a12ad859bd7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:24.480846 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 566/1113] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:07:24 2022] GET /identity/v3/roles/a74bf01dd66f49f5988a197da269c822 => generated 223 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:24.487156 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47005b7-f1a8-4383-bb6b-39707c3c981a None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:24.590477 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47005b7-f1a8-4383-bb6b-39707c3c981a None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:24.625500 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6fb8c3-0cd5-45d0-afc0-1d4b16d45050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:24.626132 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6fb8c3-0cd5-45d0-afc0-1d4b16d45050 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:24.626406 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6fb8c3-0cd5-45d0-afc0-1d4b16d45050 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:24.626665 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6fb8c3-0cd5-45d0-afc0-1d4b16d45050 None admin] PATH_INFO: `/v3/users/38a3d66f881b406ba51467a444de7a55` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:24.640528 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6fb8c3-0cd5-45d0-afc0-1d4b16d45050 None admin] RBAC: Authorizing `identity:get_user(user_id=38a3d66f881b406ba51467a444de7a55)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:24.641986 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6fb8c3-0cd5-45d0-afc0-1d4b16d45050 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:24.643648 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 548/1114] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:07:24 2022] GET /identity/v3/users/38a3d66f881b406ba51467a444de7a55 => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:24.650213 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693bbd83-df28-4651-9d2d-5a12ad859bd7 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:24.757665 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693bbd83-df28-4651-9d2d-5a12ad859bd7 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:24.792558 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1e91c1-b19c-4841-90ca-635decbce914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:24.793155 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e91c1-b19c-4841-90ca-635decbce914 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:24.793432 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e91c1-b19c-4841-90ca-635decbce914 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:24.793696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e91c1-b19c-4841-90ca-635decbce914 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:24.847686 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:24.864955 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1e91c1-b19c-4841-90ca-635decbce914 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:24.866432 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1e91c1-b19c-4841-90ca-635decbce914 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:24.971956 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 567/1115] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:07:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 05:07:24.978329 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6fb8c3-0cd5-45d0-afc0-1d4b16d45050 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:25.027534 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:25.093580 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6fb8c3-0cd5-45d0-afc0-1d4b16d45050 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:25.132388 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdadcf17-e3ca-4243-959b-e9952e772c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:25.132934 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdadcf17-e3ca-4243-959b-e9952e772c8e None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:25.133205 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdadcf17-e3ca-4243-959b-e9952e772c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:25.133469 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdadcf17-e3ca-4243-959b-e9952e772c8e None admin] PATH_INFO: `/v3/projects/6647c4a49c894179b981c3f9e904b7c4` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:25.134755 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdadcf17-e3ca-4243-959b-e9952e772c8e None admin] RBAC: Authorizing `identity:get_project(project_id=6647c4a49c894179b981c3f9e904b7c4)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:25.136292 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdadcf17-e3ca-4243-959b-e9952e772c8e None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:25.137900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 549/1116] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:07:24 2022] GET /identity/v3/projects/6647c4a49c894179b981c3f9e904b7c4 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:25.144273 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1e91c1-b19c-4841-90ca-635decbce914 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:25.227755 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:25.246867 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1e91c1-b19c-4841-90ca-635decbce914 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:25.281539 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca661b-cd81-4314-9d43-7db4335870cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:25.282147 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca661b-cd81-4314-9d43-7db4335870cc None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:25.282422 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca661b-cd81-4314-9d43-7db4335870cc None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:25.282685 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca661b-cd81-4314-9d43-7db4335870cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:25.283392 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ca661b-cd81-4314-9d43-7db4335870cc None admin] RBAC: Adding query filter params (role.id=a74bf01dd66f49f5988a197da269c822, scope.project.id=6647c4a49c894179b981c3f9e904b7c4, user.id=38a3d66f881b406ba51467a444de7a55) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:25.283702 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ca661b-cd81-4314-9d43-7db4335870cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:25.285106 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ca661b-cd81-4314-9d43-7db4335870cc None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:25.290597 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 568/1117] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:07:25 2022] GET /identity/v3/role_assignments?user.id=38a3d66f881b406ba51467a444de7a55&scope.project.id=6647c4a49c894179b981c3f9e904b7c4&role.id=a74bf01dd66f49f5988a197da269c822 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:27.152390 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19c371-8428-4b10-989b-1196e88acaa8 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:27.153266 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19c371-8428-4b10-989b-1196e88acaa8 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:27.153266 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19c371-8428-4b10-989b-1196e88acaa8 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:27.155417 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 550/1118] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:07:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:07:27.164126 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9dd1d-b4b7-4141-9159-e12ab4191ac7 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:27.164565 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9dd1d-b4b7-4141-9159-e12ab4191ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:27.164670 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9dd1d-b4b7-4141-9159-e12ab4191ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:27.186348 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-50b9dd1d-b4b7-4141-9159-e12ab4191ac7 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:07:27.187527 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-50b9dd1d-b4b7-4141-9159-e12ab4191ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:07:27.257060 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 569/1119] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:07:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:07:27.516472 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c19c371-8428-4b10-989b-1196e88acaa8 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:27.626094 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c19c371-8428-4b10-989b-1196e88acaa8 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:27.628375 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:27.669553 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90660c1-ee11-4675-b5ea-3dca7157221d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:27.670224 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90660c1-ee11-4675-b5ea-3dca7157221d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:27.670500 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90660c1-ee11-4675-b5ea-3dca7157221d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:27.670770 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90660c1-ee11-4675-b5ea-3dca7157221d None admin] PATH_INFO: `/v3/roles/a74bf01dd66f49f5988a197da269c822` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:27.672239 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90660c1-ee11-4675-b5ea-3dca7157221d None admin] RBAC: Authorizing `identity:get_role(role_id=a74bf01dd66f49f5988a197da269c822)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:27.673634 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90660c1-ee11-4675-b5ea-3dca7157221d None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:27.675048 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 551/1120] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:07:27 2022] GET /identity/v3/roles/a74bf01dd66f49f5988a197da269c822 => generated 223 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:27.681135 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9dd1d-b4b7-4141-9159-e12ab4191ac7 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:27.769218 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:27.792909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9dd1d-b4b7-4141-9159-e12ab4191ac7 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:27.828434 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf10733e-13cd-4594-9815-f685b675e653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:27.829043 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf10733e-13cd-4594-9815-f685b675e653 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:27.829318 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf10733e-13cd-4594-9815-f685b675e653 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:27.829585 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf10733e-13cd-4594-9815-f685b675e653 None admin] PATH_INFO: `/v3/users/38a3d66f881b406ba51467a444de7a55` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:27.830745 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf10733e-13cd-4594-9815-f685b675e653 None admin] RBAC: Authorizing `identity:get_user(user_id=38a3d66f881b406ba51467a444de7a55)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:27.832175 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf10733e-13cd-4594-9815-f685b675e653 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:27.833746 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 570/1121] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:07:27 2022] GET /identity/v3/users/38a3d66f881b406ba51467a444de7a55 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:27.840259 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90660c1-ee11-4675-b5ea-3dca7157221d None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:27.955866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90660c1-ee11-4675-b5ea-3dca7157221d None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:27.991008 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc41ab-560d-4d2d-a21d-2043d0c68e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:27.991619 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc41ab-560d-4d2d-a21d-2043d0c68e8a None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:27.992216 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc41ab-560d-4d2d-a21d-2043d0c68e8a None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:27.992492 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc41ab-560d-4d2d-a21d-2043d0c68e8a None admin] PATH_INFO: `/v3/projects/6647c4a49c894179b981c3f9e904b7c4` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:27.993646 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc41ab-560d-4d2d-a21d-2043d0c68e8a None admin] RBAC: Authorizing `identity:get_project(project_id=6647c4a49c894179b981c3f9e904b7c4)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:27.995124 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc41ab-560d-4d2d-a21d-2043d0c68e8a None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:27.996756 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 552/1122] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:07:27 2022] GET /identity/v3/projects/6647c4a49c894179b981c3f9e904b7c4 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:28.006047 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf10733e-13cd-4594-9815-f685b675e653 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:28.110574 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf10733e-13cd-4594-9815-f685b675e653 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:28.151402 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b2e86b-2a22-4ab4-8dfe-f64f02cbbb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:28.152069 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2e86b-2a22-4ab4-8dfe-f64f02cbbb97 None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:28.152339 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2e86b-2a22-4ab4-8dfe-f64f02cbbb97 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:28.152601 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2e86b-2a22-4ab4-8dfe-f64f02cbbb97 None admin] PATH_INFO: `/v3/projects/6647c4a49c894179b981c3f9e904b7c4/users/38a3d66f881b406ba51467a444de7a55/roles/a74bf01dd66f49f5988a197da269c822` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:28.155097 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b2e86b-2a22-4ab4-8dfe-f64f02cbbb97 None admin] RBAC: Authorizing `identity:create_grant(project_id=6647c4a49c894179b981c3f9e904b7c4, user_id=38a3d66f881b406ba51467a444de7a55, role_id=a74bf01dd66f49f5988a197da269c822)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:28.156522 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b2e86b-2a22-4ab4-8dfe-f64f02cbbb97 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:28.172246 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 571/1123] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 05:07:28 2022] PUT /identity/v3/projects/6647c4a49c894179b981c3f9e904b7c4/users/38a3d66f881b406ba51467a444de7a55/roles/a74bf01dd66f49f5988a197da269c822 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:07:30.022453 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592fa448-339d-422a-a327-7ed2c317de5c None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:30.023005 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592fa448-339d-422a-a327-7ed2c317de5c None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:30.023005 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592fa448-339d-422a-a327-7ed2c317de5c None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:30.024395 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 553/1124] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:07:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:07:30.030725 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7588d42e-4e70-423c-8579-883dfe957336 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:30.031481 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7588d42e-4e70-423c-8579-883dfe957336 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:30.031629 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7588d42e-4e70-423c-8579-883dfe957336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:30.049233 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:30.056176 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-7588d42e-4e70-423c-8579-883dfe957336 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:07:30.058704 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7588d42e-4e70-423c-8579-883dfe957336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:07:30.134188 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 572/1125] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:07:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:07:30.398461 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592fa448-339d-422a-a327-7ed2c317de5c None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:30.443348 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:30.511812 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592fa448-339d-422a-a327-7ed2c317de5c None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:30.547196 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac21a50-b5b5-40e6-be54-7e826d824eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:30.547817 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac21a50-b5b5-40e6-be54-7e826d824eb5 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:30.548096 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac21a50-b5b5-40e6-be54-7e826d824eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:30.548365 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac21a50-b5b5-40e6-be54-7e826d824eb5 None admin] PATH_INFO: `/v3/roles/a74bf01dd66f49f5988a197da269c822` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:30.549880 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac21a50-b5b5-40e6-be54-7e826d824eb5 None admin] RBAC: Authorizing `identity:get_role(role_id=a74bf01dd66f49f5988a197da269c822)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:30.551317 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac21a50-b5b5-40e6-be54-7e826d824eb5 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:30.552635 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 554/1126] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:07:30 2022] GET /identity/v3/roles/a74bf01dd66f49f5988a197da269c822 => generated 223 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:30.560356 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7588d42e-4e70-423c-8579-883dfe957336 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:30.663167 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7588d42e-4e70-423c-8579-883dfe957336 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:30.700351 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f5fa2-5810-4149-b136-733768b8fa70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:30.700971 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f5fa2-5810-4149-b136-733768b8fa70 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:30.701242 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f5fa2-5810-4149-b136-733768b8fa70 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:30.701507 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f5fa2-5810-4149-b136-733768b8fa70 None admin] PATH_INFO: `/v3/users/38a3d66f881b406ba51467a444de7a55` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:30.702701 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273f5fa2-5810-4149-b136-733768b8fa70 None admin] RBAC: Authorizing `identity:get_user(user_id=38a3d66f881b406ba51467a444de7a55)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:30.704126 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273f5fa2-5810-4149-b136-733768b8fa70 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:30.705718 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 573/1127] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:07:30 2022] GET /identity/v3/users/38a3d66f881b406ba51467a444de7a55 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:30.711783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac21a50-b5b5-40e6-be54-7e826d824eb5 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:30.816147 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac21a50-b5b5-40e6-be54-7e826d824eb5 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:30.825301 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:30.862174 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226eb5b8-8b98-49aa-8121-aee8d0c03486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:30.862770 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226eb5b8-8b98-49aa-8121-aee8d0c03486 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:30.863047 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226eb5b8-8b98-49aa-8121-aee8d0c03486 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:30.863315 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226eb5b8-8b98-49aa-8121-aee8d0c03486 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:30.932150 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226eb5b8-8b98-49aa-8121-aee8d0c03486 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:30.934518 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226eb5b8-8b98-49aa-8121-aee8d0c03486 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:31.055697 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 555/1128] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:07:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 05:07:31.062109 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f5fa2-5810-4149-b136-733768b8fa70 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:31.068259 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:31.168162 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f5fa2-5810-4149-b136-733768b8fa70 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:31.204499 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4892e853-3bdc-46d6-85de-f9546f9280b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:31.205031 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892e853-3bdc-46d6-85de-f9546f9280b9 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:31.205305 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892e853-3bdc-46d6-85de-f9546f9280b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:31.205575 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892e853-3bdc-46d6-85de-f9546f9280b9 None admin] PATH_INFO: `/v3/projects/6647c4a49c894179b981c3f9e904b7c4` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:31.206693 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4892e853-3bdc-46d6-85de-f9546f9280b9 None admin] RBAC: Authorizing `identity:get_project(project_id=6647c4a49c894179b981c3f9e904b7c4)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:31.208194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4892e853-3bdc-46d6-85de-f9546f9280b9 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:31.209836 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 574/1129] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:07:31 2022] GET /identity/v3/projects/6647c4a49c894179b981c3f9e904b7c4 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:07:31.218309 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226eb5b8-8b98-49aa-8121-aee8d0c03486 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:31.232012 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:31.338959 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226eb5b8-8b98-49aa-8121-aee8d0c03486 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:31.375304 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13d93a-b19f-4119-9a02-074359e37ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:31.375900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13d93a-b19f-4119-9a02-074359e37ea3 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:31.376175 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13d93a-b19f-4119-9a02-074359e37ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:31.376444 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13d93a-b19f-4119-9a02-074359e37ea3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:31.377167 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d13d93a-b19f-4119-9a02-074359e37ea3 None admin] RBAC: Adding query filter params (role.id=a74bf01dd66f49f5988a197da269c822, scope.project.id=6647c4a49c894179b981c3f9e904b7c4, user.id=38a3d66f881b406ba51467a444de7a55) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:31.377466 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d13d93a-b19f-4119-9a02-074359e37ea3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:31.378943 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d13d93a-b19f-4119-9a02-074359e37ea3 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:31.384831 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 556/1130] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:07:31 2022] GET /identity/v3/role_assignments?user.id=38a3d66f881b406ba51467a444de7a55&scope.project.id=6647c4a49c894179b981c3f9e904b7c4&role.id=a74bf01dd66f49f5988a197da269c822 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:33.279229 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a054b4a-1e14-4024-9140-3fa224734674 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:33.279781 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a054b4a-1e14-4024-9140-3fa224734674 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:33.279781 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a054b4a-1e14-4024-9140-3fa224734674 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:33.281841 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 575/1131] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:07:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:07:33.289815 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77394fdd-15d2-42d4-b1e1-d6b8c6c7353e None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:33.290305 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77394fdd-15d2-42d4-b1e1-d6b8c6c7353e None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:33.290396 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77394fdd-15d2-42d4-b1e1-d6b8c6c7353e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:33.308886 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-77394fdd-15d2-42d4-b1e1-d6b8c6c7353e None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:07:33.310058 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-77394fdd-15d2-42d4-b1e1-d6b8c6c7353e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:07:33.379855 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 557/1132] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:07:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 05:07:33.644112 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a054b4a-1e14-4024-9140-3fa224734674 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:33.721394 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:33.761621 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a054b4a-1e14-4024-9140-3fa224734674 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:33.801501 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4712b68-458c-4c66-b149-aace8547cf20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:33.802165 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4712b68-458c-4c66-b149-aace8547cf20 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:33.802432 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4712b68-458c-4c66-b149-aace8547cf20 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:33.802790 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4712b68-458c-4c66-b149-aace8547cf20 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:33.804080 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4712b68-458c-4c66-b149-aace8547cf20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:33.805686 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4712b68-458c-4c66-b149-aace8547cf20 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:33.807262 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 576/1133] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:07:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:07:33.813366 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77394fdd-15d2-42d4-b1e1-d6b8c6c7353e None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:33.815704 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:33.923073 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77394fdd-15d2-42d4-b1e1-d6b8c6c7353e None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:33.960871 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571652c-2cdc-49ca-a8f7-2749e4f8e1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:33.961465 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571652c-2cdc-49ca-a8f7-2749e4f8e1aa None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:33.961740 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571652c-2cdc-49ca-a8f7-2749e4f8e1aa None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:33.962038 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571652c-2cdc-49ca-a8f7-2749e4f8e1aa None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:33.962669 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5571652c-2cdc-49ca-a8f7-2749e4f8e1aa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:33.964127 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5571652c-2cdc-49ca-a8f7-2749e4f8e1aa None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:33.992638 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 558/1134] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Aug 31 05:07:33 2022] POST /identity/v3/projects => generated 311 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:07:35.894666 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680a38f-17ec-44c7-9233-0f3b1983eecd None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:35.895496 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680a38f-17ec-44c7-9233-0f3b1983eecd None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:35.895496 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680a38f-17ec-44c7-9233-0f3b1983eecd None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:35.897539 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 577/1135] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:07:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:07:35.903812 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb154ab-28bb-404f-8a2b-5443392f48b3 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:35.904502 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb154ab-28bb-404f-8a2b-5443392f48b3 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:35.904591 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb154ab-28bb-404f-8a2b-5443392f48b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:35.933877 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-5eb154ab-28bb-404f-8a2b-5443392f48b3 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:07:35.935167 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-5eb154ab-28bb-404f-8a2b-5443392f48b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:07:35.988491 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:36.007225 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 559/1136] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:07:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:07:36.272371 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8680a38f-17ec-44c7-9233-0f3b1983eecd None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:36.386797 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8680a38f-17ec-44c7-9233-0f3b1983eecd None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:36.420680 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aad525-4525-47b7-9773-d98d441b9e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:36.421286 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aad525-4525-47b7-9773-d98d441b9e0c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:36.421557 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aad525-4525-47b7-9773-d98d441b9e0c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:36.421821 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aad525-4525-47b7-9773-d98d441b9e0c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:36.423103 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aad525-4525-47b7-9773-d98d441b9e0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:36.424581 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aad525-4525-47b7-9773-d98d441b9e0c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:36.426224 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 578/1137] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:07:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:36.432072 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb154ab-28bb-404f-8a2b-5443392f48b3 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:36.555657 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb154ab-28bb-404f-8a2b-5443392f48b3 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:36.591805 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32bc4b7-030d-478f-a089-1951f5bd292f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:36.592416 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32bc4b7-030d-478f-a089-1951f5bd292f None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:36.592694 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32bc4b7-030d-478f-a089-1951f5bd292f None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:36.592966 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32bc4b7-030d-478f-a089-1951f5bd292f None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:36.593610 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32bc4b7-030d-478f-a089-1951f5bd292f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:36.595076 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32bc4b7-030d-478f-a089-1951f5bd292f None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:36.619718 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 560/1138] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Aug 31 05:07:36 2022] POST /identity/v3/users => generated 286 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:07:38.513223 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c710b-54fc-4921-81b0-9ae00fcd82da None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:38.513785 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c710b-54fc-4921-81b0-9ae00fcd82da None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:38.513785 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c710b-54fc-4921-81b0-9ae00fcd82da None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:38.515238 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 579/1139] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:07:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:07:38.521654 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa95d6-5bb4-4575-beaa-3a4dc1becdbd None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:38.522461 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa95d6-5bb4-4575-beaa-3a4dc1becdbd None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:38.522599 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa95d6-5bb4-4575-beaa-3a4dc1becdbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:38.549190 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-3ffa95d6-5bb4-4575-beaa-3a4dc1becdbd None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:07:38.550406 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3ffa95d6-5bb4-4575-beaa-3a4dc1becdbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:07:38.586836 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:38.624929 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 561/1140] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:07:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:07:38.891968 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c710b-54fc-4921-81b0-9ae00fcd82da None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:38.906723 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:39.006565 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c710b-54fc-4921-81b0-9ae00fcd82da None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:39.039682 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1997fe76-2413-49a8-b712-bedaa3344599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:39.040290 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997fe76-2413-49a8-b712-bedaa3344599 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:39.040561 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997fe76-2413-49a8-b712-bedaa3344599 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:39.040821 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997fe76-2413-49a8-b712-bedaa3344599 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:39.051566 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1997fe76-2413-49a8-b712-bedaa3344599 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:07:39.056391 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-1997fe76-2413-49a8-b712-bedaa3344599 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:07:39.056391 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:07:39.056391 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:07:39.056391 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:07:39.056391 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:07:39.056391 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:07:39.056391 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:07:39.056391 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:07:39.056391 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:07:39.056391 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:07:39.056391 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:07:39.056391 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:07:39.056391 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:07:39.056391 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:07:39.056391 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:07:39.056391 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:07:39.057580 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:07:39.057580 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:07:39.057580 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:07:39.057580 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:07:39.057580 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:07:39.057580 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:07:39.057580 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:07:39.057580 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:07:39.057580 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:07:39.057580 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:07:39.057580 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:07:39.057580 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:07:39.057580 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:07:39.057580 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:07:39.057580 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:07:39.057580 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:07:39.058700 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:07:39.058700 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:07:39.058700 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:07:39.058700 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:07:39.058700 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:07:39.058700 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:07:39.058700 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:07:39.058700 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:07:39.058700 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:07:39.058700 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:07:39.058700 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:07:39.058700 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:07:39.058700 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 580/1141] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 05:07:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:07:39.066890 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa95d6-5bb4-4575-beaa-3a4dc1becdbd None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:39.181944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa95d6-5bb4-4575-beaa-3a4dc1becdbd None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:39.217241 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d66893f-07dd-4d5d-a563-bb00e1f190af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:39.217831 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66893f-07dd-4d5d-a563-bb00e1f190af None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:39.218133 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66893f-07dd-4d5d-a563-bb00e1f190af None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:39.218404 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66893f-07dd-4d5d-a563-bb00e1f190af None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:39.219070 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d66893f-07dd-4d5d-a563-bb00e1f190af None admin] RBAC: Adding query filter params (name=admin) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:39.219372 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d66893f-07dd-4d5d-a563-bb00e1f190af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:39.220738 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d66893f-07dd-4d5d-a563-bb00e1f190af None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:39.228776 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 562/1142] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:07:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:39.235479 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1997fe76-2413-49a8-b712-bedaa3344599 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:39.310851 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:39.348608 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1997fe76-2413-49a8-b712-bedaa3344599 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:39.386931 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe80bc-86b3-4302-8908-c5396f988369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:39.387542 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe80bc-86b3-4302-8908-c5396f988369 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:39.387815 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe80bc-86b3-4302-8908-c5396f988369 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:39.388075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe80bc-86b3-4302-8908-c5396f988369 None admin] PATH_INFO: `/v3/users/432658e411724130ab68e51ae31913c7` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:39.402255 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fe80bc-86b3-4302-8908-c5396f988369 None admin] RBAC: Authorizing `identity:get_user(user_id=432658e411724130ab68e51ae31913c7)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:39.403789 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fe80bc-86b3-4302-8908-c5396f988369 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:39.405380 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 581/1143] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:07:39 2022] GET /identity/v3/users/432658e411724130ab68e51ae31913c7 => generated 286 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:39.415068 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d66893f-07dd-4d5d-a563-bb00e1f190af None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:39.459956 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:39.531358 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d66893f-07dd-4d5d-a563-bb00e1f190af None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:39.568098 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4945cab-7546-471c-a4f2-7549e315355c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:39.568698 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4945cab-7546-471c-a4f2-7549e315355c None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:39.568974 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4945cab-7546-471c-a4f2-7549e315355c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:39.569240 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4945cab-7546-471c-a4f2-7549e315355c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:39.640213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4945cab-7546-471c-a4f2-7549e315355c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:39.641657 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4945cab-7546-471c-a4f2-7549e315355c None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:39.672806 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:39.743565 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 563/1144] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:07:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 05:07:39.749905 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe80bc-86b3-4302-8908-c5396f988369 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:39.861304 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe80bc-86b3-4302-8908-c5396f988369 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:39.889210 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:39.904445 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d103dc1c-60be-4ada-8b62-883a0207d4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:39.905390 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d103dc1c-60be-4ada-8b62-883a0207d4bc None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:39.905748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d103dc1c-60be-4ada-8b62-883a0207d4bc None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:39.906091 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d103dc1c-60be-4ada-8b62-883a0207d4bc None admin] PATH_INFO: `/v3/projects/db596d4ab6c8433787fdd3127d042249` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:39.907873 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d103dc1c-60be-4ada-8b62-883a0207d4bc None admin] RBAC: Authorizing `identity:get_project(project_id=db596d4ab6c8433787fdd3127d042249)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:39.910188 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d103dc1c-60be-4ada-8b62-883a0207d4bc None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:39.912823 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 582/1145] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:07:39 2022] GET /identity/v3/projects/db596d4ab6c8433787fdd3127d042249 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:39.920569 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4945cab-7546-471c-a4f2-7549e315355c None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:40.026434 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4945cab-7546-471c-a4f2-7549e315355c None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:40.056306 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:40.062232 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e0ec66-adfb-4eca-bc15-cf77d5a88da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:40.062827 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e0ec66-adfb-4eca-bc15-cf77d5a88da7 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:40.063103 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e0ec66-adfb-4eca-bc15-cf77d5a88da7 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:40.063371 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e0ec66-adfb-4eca-bc15-cf77d5a88da7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:40.064147 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e0ec66-adfb-4eca-bc15-cf77d5a88da7 None admin] RBAC: Adding query filter params (role.id=c3d5589dd2864be9a0f26f5802d874cd, scope.project.id=db596d4ab6c8433787fdd3127d042249, user.id=432658e411724130ab68e51ae31913c7) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:40.064477 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e0ec66-adfb-4eca-bc15-cf77d5a88da7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:40.065946 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e0ec66-adfb-4eca-bc15-cf77d5a88da7 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:40.071599 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 564/1146] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:07:39 2022] GET /identity/v3/role_assignments?user.id=432658e411724130ab68e51ae31913c7&scope.project.id=db596d4ab6c8433787fdd3127d042249&role.id=c3d5589dd2864be9a0f26f5802d874cd => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:41.923008 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b0b6d-7c35-4796-aa67-7ff637c9c68e None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:41.923551 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b0b6d-7c35-4796-aa67-7ff637c9c68e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:41.923551 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b0b6d-7c35-4796-aa67-7ff637c9c68e None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:41.924807 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 583/1147] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:07:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:07:41.931251 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9332b1-ac34-4119-a32e-d8e846720eb3 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:41.932008 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9332b1-ac34-4119-a32e-d8e846720eb3 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:41.932254 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9332b1-ac34-4119-a32e-d8e846720eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:41.952042 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-6b9332b1-ac34-4119-a32e-d8e846720eb3 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:07:41.953249 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-6b9332b1-ac34-4119-a32e-d8e846720eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:07:42.023347 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 565/1148] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:07:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:07:42.298151 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b0b6d-7c35-4796-aa67-7ff637c9c68e None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:42.423892 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b0b6d-7c35-4796-aa67-7ff637c9c68e None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:42.459076 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0afbfd1-a249-4119-8a5d-0f968c7da384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:42.459693 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0afbfd1-a249-4119-8a5d-0f968c7da384 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:42.459969 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0afbfd1-a249-4119-8a5d-0f968c7da384 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:42.460238 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0afbfd1-a249-4119-8a5d-0f968c7da384 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:42.471122 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0afbfd1-a249-4119-8a5d-0f968c7da384 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:07:42.476108 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-c0afbfd1-a249-4119-8a5d-0f968c7da384 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:07:42.476108 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:07:42.476108 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:07:42.476108 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:07:42.476108 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:07:42.476108 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:07:42.476108 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:07:42.476108 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:07:42.476108 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:07:42.476108 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:07:42.476108 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:07:42.476108 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:07:42.476108 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:07:42.476108 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:07:42.476108 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:07:42.476108 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:07:42.477269 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:07:42.477269 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:07:42.477269 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:07:42.477269 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:07:42.477269 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:07:42.477269 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:07:42.477269 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:07:42.477269 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:07:42.477269 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:07:42.477269 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:07:42.477269 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:07:42.477269 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:07:42.477269 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:07:42.477269 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:07:42.477269 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:07:42.477269 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:07:42.478312 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:07:42.478312 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:07:42.478312 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:07:42.478312 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:07:42.478312 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:07:42.478312 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:07:42.478312 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:07:42.478312 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:07:42.478312 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:07:42.478312 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:07:42.478312 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:07:42.478312 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:07:42.482156 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 584/1149] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 05:07:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:07:42.483347 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9332b1-ac34-4119-a32e-d8e846720eb3 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:42.593014 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9332b1-ac34-4119-a32e-d8e846720eb3 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:42.620041 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:42.630573 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1300ddc4-167e-4ae1-8f8c-11fe5e79d209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:42.631161 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1300ddc4-167e-4ae1-8f8c-11fe5e79d209 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:42.631431 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1300ddc4-167e-4ae1-8f8c-11fe5e79d209 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:42.631693 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1300ddc4-167e-4ae1-8f8c-11fe5e79d209 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:42.632343 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1300ddc4-167e-4ae1-8f8c-11fe5e79d209 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:42.632633 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1300ddc4-167e-4ae1-8f8c-11fe5e79d209 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:42.634025 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1300ddc4-167e-4ae1-8f8c-11fe5e79d209 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:42.643976 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 566/1150] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:07:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:07:42.652636 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0afbfd1-a249-4119-8a5d-0f968c7da384 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:42.711501 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:42.769614 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0afbfd1-a249-4119-8a5d-0f968c7da384 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:42.811727 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b3c8b-8e77-4e8d-8342-2ae8bc0d9ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:42.812337 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b3c8b-8e77-4e8d-8342-2ae8bc0d9ee6 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:42.812612 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b3c8b-8e77-4e8d-8342-2ae8bc0d9ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:42.812878 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b3c8b-8e77-4e8d-8342-2ae8bc0d9ee6 None admin] PATH_INFO: `/v3/users/432658e411724130ab68e51ae31913c7` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:42.813993 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903b3c8b-8e77-4e8d-8342-2ae8bc0d9ee6 None admin] RBAC: Authorizing `identity:get_user(user_id=432658e411724130ab68e51ae31913c7)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:42.815450 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903b3c8b-8e77-4e8d-8342-2ae8bc0d9ee6 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:42.816988 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 585/1151] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:07:42 2022] GET /identity/v3/users/432658e411724130ab68e51ae31913c7 => generated 286 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:42.825085 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1300ddc4-167e-4ae1-8f8c-11fe5e79d209 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:42.930080 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1300ddc4-167e-4ae1-8f8c-11fe5e79d209 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:42.965384 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738032ec-7540-4e60-8a4d-d4e2cb5e75c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:42.965959 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738032ec-7540-4e60-8a4d-d4e2cb5e75c7 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:42.966282 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738032ec-7540-4e60-8a4d-d4e2cb5e75c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:42.966545 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738032ec-7540-4e60-8a4d-d4e2cb5e75c7 None admin] PATH_INFO: `/v3/projects/db596d4ab6c8433787fdd3127d042249` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:42.967786 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738032ec-7540-4e60-8a4d-d4e2cb5e75c7 None admin] RBAC: Authorizing `identity:get_project(project_id=db596d4ab6c8433787fdd3127d042249)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:42.969213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738032ec-7540-4e60-8a4d-d4e2cb5e75c7 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:42.970857 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 567/1152] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:07:42 2022] GET /identity/v3/projects/db596d4ab6c8433787fdd3127d042249 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:42.979696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b3c8b-8e77-4e8d-8342-2ae8bc0d9ee6 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:43.086731 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b3c8b-8e77-4e8d-8342-2ae8bc0d9ee6 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:43.115816 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:43.129269 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e39937-4327-43a5-b216-4a50e39f0da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:43.129915 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e39937-4327-43a5-b216-4a50e39f0da6 None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:43.130221 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e39937-4327-43a5-b216-4a50e39f0da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:43.130490 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e39937-4327-43a5-b216-4a50e39f0da6 None admin] PATH_INFO: `/v3/projects/db596d4ab6c8433787fdd3127d042249/users/432658e411724130ab68e51ae31913c7/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:43.132476 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e39937-4327-43a5-b216-4a50e39f0da6 None admin] RBAC: Authorizing `identity:create_grant(project_id=db596d4ab6c8433787fdd3127d042249, user_id=432658e411724130ab68e51ae31913c7, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:43.133914 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e39937-4327-43a5-b216-4a50e39f0da6 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:43.148958 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 586/1153] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 05:07:42 2022] PUT /identity/v3/projects/db596d4ab6c8433787fdd3127d042249/users/432658e411724130ab68e51ae31913c7/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:07:44.987165 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134390d-0398-41cc-ba38-095568220c57 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:44.988040 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134390d-0398-41cc-ba38-095568220c57 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:44.988040 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134390d-0398-41cc-ba38-095568220c57 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:44.990073 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 568/1154] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:07:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:07:44.998329 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e67ad-48ba-4710-a540-67ad8b04e483 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:44.999022 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e67ad-48ba-4710-a540-67ad8b04e483 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:44.999140 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e67ad-48ba-4710-a540-67ad8b04e483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:45.022450 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e15e67ad-48ba-4710-a540-67ad8b04e483 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:07:45.023609 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e15e67ad-48ba-4710-a540-67ad8b04e483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:07:45.098053 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 587/1155] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:07:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:07:45.359477 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5134390d-0398-41cc-ba38-095568220c57 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:45.421830 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:45.473692 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5134390d-0398-41cc-ba38-095568220c57 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:45.511736 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e200bce9-d2e2-40b1-8a04-9541f41383cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:45.512353 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200bce9-d2e2-40b1-8a04-9541f41383cc None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:45.512616 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200bce9-d2e2-40b1-8a04-9541f41383cc None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:45.512895 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200bce9-d2e2-40b1-8a04-9541f41383cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:45.523760 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e200bce9-d2e2-40b1-8a04-9541f41383cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:07:45.528573 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-e200bce9-d2e2-40b1-8a04-9541f41383cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:07:45.528573 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:07:45.528573 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:07:45.528573 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:07:45.528573 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:07:45.528573 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:07:45.528573 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:07:45.528573 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:07:45.528573 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:07:45.528573 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:07:45.528573 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:07:45.528573 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:07:45.528573 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:07:45.528573 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:07:45.528573 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:07:45.528573 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:07:45.529650 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:07:45.529650 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:07:45.529650 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:07:45.529650 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:07:45.529650 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:07:45.529650 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:07:45.529650 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:07:45.529650 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:07:45.529650 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:07:45.529650 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:07:45.529650 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:07:45.529650 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:07:45.529650 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:07:45.529650 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:07:45.529650 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:07:45.529650 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:07:45.530821 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:07:45.530821 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:07:45.530821 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:07:45.530821 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:07:45.530821 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:07:45.530821 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:07:45.530821 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:07:45.530821 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:07:45.530821 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:07:45.530821 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:07:45.530821 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:07:45.530821 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:07:45.530821 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 569/1156] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 05:07:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:07:45.535700 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e67ad-48ba-4710-a540-67ad8b04e483 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:45.637654 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e67ad-48ba-4710-a540-67ad8b04e483 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:45.651016 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:45.674888 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097af3a5-5ff1-466b-9252-5eece192a20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:45.675498 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097af3a5-5ff1-466b-9252-5eece192a20a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:45.675774 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097af3a5-5ff1-466b-9252-5eece192a20a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:45.676042 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097af3a5-5ff1-466b-9252-5eece192a20a None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:45.676705 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097af3a5-5ff1-466b-9252-5eece192a20a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:45.677004 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097af3a5-5ff1-466b-9252-5eece192a20a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:45.678402 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097af3a5-5ff1-466b-9252-5eece192a20a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:45.686569 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 588/1157] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:07:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:45.695367 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e200bce9-d2e2-40b1-8a04-9541f41383cc None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:45.796319 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e200bce9-d2e2-40b1-8a04-9541f41383cc None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:45.812771 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:45.831964 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b04aa-5b0c-48b2-881e-0cf59f463da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:45.832570 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b04aa-5b0c-48b2-881e-0cf59f463da4 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:45.832841 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b04aa-5b0c-48b2-881e-0cf59f463da4 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:45.833107 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b04aa-5b0c-48b2-881e-0cf59f463da4 None admin] PATH_INFO: `/v3/users/432658e411724130ab68e51ae31913c7` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:45.834278 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b04aa-5b0c-48b2-881e-0cf59f463da4 None admin] RBAC: Authorizing `identity:get_user(user_id=432658e411724130ab68e51ae31913c7)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:45.835717 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b04aa-5b0c-48b2-881e-0cf59f463da4 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:45.837258 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 570/1158] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:07:45 2022] GET /identity/v3/users/432658e411724130ab68e51ae31913c7 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:45.843938 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097af3a5-5ff1-466b-9252-5eece192a20a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:45.963945 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097af3a5-5ff1-466b-9252-5eece192a20a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:46.001158 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6949b34c-cad5-4c4e-a134-56fa7c11a676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:46.001778 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949b34c-cad5-4c4e-a134-56fa7c11a676 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:46.002047 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949b34c-cad5-4c4e-a134-56fa7c11a676 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:46.002319 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949b34c-cad5-4c4e-a134-56fa7c11a676 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:46.045182 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:46.081919 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6949b34c-cad5-4c4e-a134-56fa7c11a676 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:46.083402 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6949b34c-cad5-4c4e-a134-56fa7c11a676 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:46.189329 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 589/1159] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:07:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 05:07:46.198569 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b04aa-5b0c-48b2-881e-0cf59f463da4 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:46.314734 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b04aa-5b0c-48b2-881e-0cf59f463da4 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:46.351014 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b0db6b-9bc7-4d8b-b7d1-d8237826b382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:46.351628 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0db6b-9bc7-4d8b-b7d1-d8237826b382 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:46.351900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0db6b-9bc7-4d8b-b7d1-d8237826b382 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:46.352171 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0db6b-9bc7-4d8b-b7d1-d8237826b382 None admin] PATH_INFO: `/v3/projects/db596d4ab6c8433787fdd3127d042249` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:46.353407 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b0db6b-9bc7-4d8b-b7d1-d8237826b382 None admin] RBAC: Authorizing `identity:get_project(project_id=db596d4ab6c8433787fdd3127d042249)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:46.354882 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b0db6b-9bc7-4d8b-b7d1-d8237826b382 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:46.356509 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 571/1160] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:07:46 2022] GET /identity/v3/projects/db596d4ab6c8433787fdd3127d042249 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:07:46.365898 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6949b34c-cad5-4c4e-a134-56fa7c11a676 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:46.440993 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:46.466244 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6949b34c-cad5-4c4e-a134-56fa7c11a676 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:46.513316 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd9fa8-a48d-4c5a-8174-51a9c478b2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:46.513900 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd9fa8-a48d-4c5a-8174-51a9c478b2b3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:46.514207 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd9fa8-a48d-4c5a-8174-51a9c478b2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:46.514473 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd9fa8-a48d-4c5a-8174-51a9c478b2b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:46.515185 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afd9fa8-a48d-4c5a-8174-51a9c478b2b3 None admin] RBAC: Adding query filter params (role.id=c3d5589dd2864be9a0f26f5802d874cd, scope.project.id=db596d4ab6c8433787fdd3127d042249, user.id=432658e411724130ab68e51ae31913c7) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:46.515488 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afd9fa8-a48d-4c5a-8174-51a9c478b2b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:46.516900 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afd9fa8-a48d-4c5a-8174-51a9c478b2b3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:46.522875 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 590/1161] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:07:46 2022] GET /identity/v3/role_assignments?user.id=432658e411724130ab68e51ae31913c7&scope.project.id=db596d4ab6c8433787fdd3127d042249&role.id=c3d5589dd2864be9a0f26f5802d874cd => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:48.431649 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692ffcb-671f-40a2-bf51-29cd7a0e1907 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:48.432246 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692ffcb-671f-40a2-bf51-29cd7a0e1907 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:48.432246 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692ffcb-671f-40a2-bf51-29cd7a0e1907 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:48.433619 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 572/1162] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:07:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:07:48.439972 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f5698-5fc2-40fb-bd6d-26e77032bf28 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:48.440690 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f5698-5fc2-40fb-bd6d-26e77032bf28 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:48.440833 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f5698-5fc2-40fb-bd6d-26e77032bf28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:48.468169 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-872f5698-5fc2-40fb-bd6d-26e77032bf28 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:07:48.469338 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-872f5698-5fc2-40fb-bd6d-26e77032bf28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:07:48.540309 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 591/1163] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:07:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:07:48.807327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692ffcb-671f-40a2-bf51-29cd7a0e1907 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:48.867566 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:48.917315 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692ffcb-671f-40a2-bf51-29cd7a0e1907 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:48.965945 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ba5c1-dcb9-4b10-9538-ed88aacf1307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:48.966585 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ba5c1-dcb9-4b10-9538-ed88aacf1307 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:48.966860 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ba5c1-dcb9-4b10-9538-ed88aacf1307 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:48.967130 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ba5c1-dcb9-4b10-9538-ed88aacf1307 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:49.035193 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386ba5c1-dcb9-4b10-9538-ed88aacf1307 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:49.036637 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386ba5c1-dcb9-4b10-9538-ed88aacf1307 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:49.088525 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:49.137756 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 573/1164] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:07:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 05:07:49.144103 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f5698-5fc2-40fb-bd6d-26e77032bf28 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:49.215273 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:49.249646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f5698-5fc2-40fb-bd6d-26e77032bf28 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:49.284595 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f94bbde-0ece-4930-8c26-e4dee382b465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:49.285207 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f94bbde-0ece-4930-8c26-e4dee382b465 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:49.285479 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f94bbde-0ece-4930-8c26-e4dee382b465 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:49.285740 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f94bbde-0ece-4930-8c26-e4dee382b465 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:49.292896 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f94bbde-0ece-4930-8c26-e4dee382b465 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:49.294368 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f94bbde-0ece-4930-8c26-e4dee382b465 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:49.306305 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-1f94bbde-0ece-4930-8c26-e4dee382b465 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 31 05:07:49.306305 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:07:49.306305 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 31 05:07:49.306305 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 31 05:07:49.306305 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 05:07:49.306305 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 05:07:49.306305 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 05:07:49.306305 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 05:07:49.306305 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 31 05:07:49.306305 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:07:49.306305 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 05:07:49.306305 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:07:49.306305 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:07:49.306305 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:07:49.306305 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:07:49.306305 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:07:49.306305 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:07:49.307470 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:07:49.307470 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:07:49.307470 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:07:49.307470 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:07:49.307470 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:07:49.307470 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:07:49.307470 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:07:49.307470 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:07:49.307470 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:07:49.307470 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:07:49.307470 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:07:49.307470 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:07:49.307470 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:07:49.307470 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:07:49.307470 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:07:49.307470 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:07:49.308536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:07:49.308536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:07:49.308536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:07:49.308536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:07:49.308536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:07:49.308536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:07:49.308536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:07:49.308536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:07:49.308536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:07:49.308536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:07:49.308536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 31 05:07:49.308536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:07:49.308536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 31 05:07:49.308536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:07:49.310202 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 592/1165] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Aug 31 05:07:49 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:07:49.313432 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ba5c1-dcb9-4b10-9538-ed88aacf1307 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:49.424077 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ba5c1-dcb9-4b10-9538-ed88aacf1307 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:49.459816 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa78e5-01db-4e31-8136-8ab93563608c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:49.460415 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa78e5-01db-4e31-8136-8ab93563608c None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:49.460668 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa78e5-01db-4e31-8136-8ab93563608c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:49.460931 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa78e5-01db-4e31-8136-8ab93563608c None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:49.461569 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaa78e5-01db-4e31-8136-8ab93563608c None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:49.461862 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaa78e5-01db-4e31-8136-8ab93563608c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:49.463263 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaa78e5-01db-4e31-8136-8ab93563608c None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:49.470563 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 574/1166] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Aug 31 05:07:49 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:51.344859 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebbd16-97fe-49df-a650-918767fa16b8 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:51.345700 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebbd16-97fe-49df-a650-918767fa16b8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:51.345700 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebbd16-97fe-49df-a650-918767fa16b8 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:51.347947 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 593/1167] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:07:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:07:51.354207 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f4800-abad-470d-a182-1bb2ed663ca1 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:51.354704 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f4800-abad-470d-a182-1bb2ed663ca1 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:51.354867 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f4800-abad-470d-a182-1bb2ed663ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:51.375085 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-a89f4800-abad-470d-a182-1bb2ed663ca1 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:07:51.376271 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a89f4800-abad-470d-a182-1bb2ed663ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:07:51.387379 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:51.449316 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 575/1168] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:07:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:07:51.714489 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eebbd16-97fe-49df-a650-918767fa16b8 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:51.828787 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eebbd16-97fe-49df-a650-918767fa16b8 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:51.857889 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:51.867082 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8be63d-6932-4179-a42b-00348f0af918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:51.867681 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8be63d-6932-4179-a42b-00348f0af918 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:51.867953 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8be63d-6932-4179-a42b-00348f0af918 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:51.868217 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8be63d-6932-4179-a42b-00348f0af918 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:51.868876 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8be63d-6932-4179-a42b-00348f0af918 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:51.870322 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8be63d-6932-4179-a42b-00348f0af918 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:51.888694 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 594/1169] 10.222.0.36 () {60 vars in 1138 bytes} [Wed Aug 31 05:07:51 2022] POST /identity/v3/domains => generated 267 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:07:53.817877 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e16d9-e6a8-42c4-bfd1-d9bbed643273 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:53.818779 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e16d9-e6a8-42c4-bfd1-d9bbed643273 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:53.818779 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e16d9-e6a8-42c4-bfd1-d9bbed643273 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:53.820874 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 576/1170] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:07:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:07:53.826553 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873de06e-1e15-448e-9b00-ffd1f3646995 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:53.826995 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873de06e-1e15-448e-9b00-ffd1f3646995 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:53.827087 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873de06e-1e15-448e-9b00-ffd1f3646995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:53.846607 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-873de06e-1e15-448e-9b00-ffd1f3646995 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:07:53.847782 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-873de06e-1e15-448e-9b00-ffd1f3646995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:07:53.916756 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 595/1171] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:07:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:07:54.185230 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e16d9-e6a8-42c4-bfd1-d9bbed643273 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:54.298686 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e16d9-e6a8-42c4-bfd1-d9bbed643273 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:54.334502 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7cd75a-6bca-47ee-8702-ae6a8cc50d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:54.335121 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7cd75a-6bca-47ee-8702-ae6a8cc50d5c None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:54.335400 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7cd75a-6bca-47ee-8702-ae6a8cc50d5c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:54.335669 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7cd75a-6bca-47ee-8702-ae6a8cc50d5c None admin] PATH_INFO: `/v3/domains/50ba911912b54a7695c7bbd0f410758e` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:54.347182 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7cd75a-6bca-47ee-8702-ae6a8cc50d5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=50ba911912b54a7695c7bbd0f410758e)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:54.349400 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7cd75a-6bca-47ee-8702-ae6a8cc50d5c None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:54.351977 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 577/1172] 10.222.0.36 () {56 vars in 1183 bytes} [Wed Aug 31 05:07:54 2022] GET /identity/v3/domains/50ba911912b54a7695c7bbd0f410758e => generated 267 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:54.358233 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873de06e-1e15-448e-9b00-ffd1f3646995 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:54.467397 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873de06e-1e15-448e-9b00-ffd1f3646995 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:54.484794 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:54.504002 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f47ef2a-c332-42e0-a45b-b264960a4ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:54.504604 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47ef2a-c332-42e0-a45b-b264960a4ed7 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:54.504881 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47ef2a-c332-42e0-a45b-b264960a4ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:54.505151 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47ef2a-c332-42e0-a45b-b264960a4ed7 None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:54.505798 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f47ef2a-c332-42e0-a45b-b264960a4ed7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:54.507279 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f47ef2a-c332-42e0-a45b-b264960a4ed7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:54.537339 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 596/1173] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Aug 31 05:07:54 2022] POST /identity/v3/projects => generated 361 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:07:56.447307 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006e056d-2935-483e-bd66-321799cfab5b None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:56.447966 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006e056d-2935-483e-bd66-321799cfab5b None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:56.447966 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006e056d-2935-483e-bd66-321799cfab5b None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:56.449350 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 578/1174] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:07:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:07:56.455639 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf373fd-711f-425f-9dd8-7247416e7d0e None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:56.456070 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf373fd-711f-425f-9dd8-7247416e7d0e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:56.456196 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf373fd-711f-425f-9dd8-7247416e7d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:56.484301 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-aaf373fd-711f-425f-9dd8-7247416e7d0e None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:07:56.485476 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-aaf373fd-711f-425f-9dd8-7247416e7d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:07:56.555176 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 597/1175] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:07:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:07:56.829315 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006e056d-2935-483e-bd66-321799cfab5b None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:56.840111 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:56.940176 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006e056d-2935-483e-bd66-321799cfab5b None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:56.976068 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b933f8-edb0-4f07-96c5-1091481fb63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:56.976668 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b933f8-edb0-4f07-96c5-1091481fb63e None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:56.976935 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b933f8-edb0-4f07-96c5-1091481fb63e None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:56.977288 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b933f8-edb0-4f07-96c5-1091481fb63e None admin] PATH_INFO: `/v3/domains/50ba911912b54a7695c7bbd0f410758e` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:56.978462 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b933f8-edb0-4f07-96c5-1091481fb63e None admin] RBAC: Authorizing `identity:get_domain(domain_id=50ba911912b54a7695c7bbd0f410758e)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:56.979972 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b933f8-edb0-4f07-96c5-1091481fb63e None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:56.981510 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 579/1176] 10.222.0.36 () {56 vars in 1183 bytes} [Wed Aug 31 05:07:56 2022] GET /identity/v3/domains/50ba911912b54a7695c7bbd0f410758e => generated 267 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:56.987334 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf373fd-711f-425f-9dd8-7247416e7d0e None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:57.106445 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:57.109284 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf373fd-711f-425f-9dd8-7247416e7d0e None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:57.145868 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6172ad6-a8a3-4def-bd81-e049f2c2fd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:57.146491 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6172ad6-a8a3-4def-bd81-e049f2c2fd5f None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:57.146764 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6172ad6-a8a3-4def-bd81-e049f2c2fd5f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:57.147029 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6172ad6-a8a3-4def-bd81-e049f2c2fd5f None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:57.147668 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6172ad6-a8a3-4def-bd81-e049f2c2fd5f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:57.149085 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6172ad6-a8a3-4def-bd81-e049f2c2fd5f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:57.171833 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 598/1177] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Aug 31 05:07:56 2022] POST /identity/v3/users => generated 311 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:07:59.093582 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841db8c7-a43d-4ea5-909e-c95854f3fc14 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:59.094182 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841db8c7-a43d-4ea5-909e-c95854f3fc14 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:59.094182 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841db8c7-a43d-4ea5-909e-c95854f3fc14 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:59.095471 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 580/1178] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:07:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:07:59.101895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875240b-a6bf-4ef4-b768-c0d262a3bc46 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:59.102689 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875240b-a6bf-4ef4-b768-c0d262a3bc46 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:59.102839 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875240b-a6bf-4ef4-b768-c0d262a3bc46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:59.128565 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-8875240b-a6bf-4ef4-b768-c0d262a3bc46 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:07:59.129740 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8875240b-a6bf-4ef4-b768-c0d262a3bc46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:07:59.200885 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 599/1179] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:07:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:07:59.468802 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841db8c7-a43d-4ea5-909e-c95854f3fc14 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:59.556158 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:59.583748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841db8c7-a43d-4ea5-909e-c95854f3fc14 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:59.619080 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7551a6-1960-452c-b9c0-59b66d0f6db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:59.619688 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7551a6-1960-452c-b9c0-59b66d0f6db2 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:59.620004 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7551a6-1960-452c-b9c0-59b66d0f6db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:59.620268 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7551a6-1960-452c-b9c0-59b66d0f6db2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:59.630728 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7551a6-1960-452c-b9c0-59b66d0f6db2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:07:59.635393 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-bd7551a6-1960-452c-b9c0-59b66d0f6db2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:07:59.635393 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:07:59.635393 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:07:59.635393 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:07:59.635393 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:07:59.635393 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:07:59.635393 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:07:59.635393 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:07:59.635393 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:07:59.635393 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:07:59.635393 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:07:59.635393 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:07:59.635393 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:07:59.635393 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:07:59.635393 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:07:59.635393 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:07:59.636601 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:07:59.636601 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:07:59.636601 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:07:59.636601 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:07:59.636601 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:07:59.636601 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:07:59.636601 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:07:59.636601 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:07:59.636601 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:07:59.636601 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:07:59.636601 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:07:59.636601 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:07:59.636601 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:07:59.636601 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:07:59.636601 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:07:59.636601 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:07:59.638049 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:07:59.638049 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:07:59.638049 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:07:59.638049 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:07:59.638049 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:07:59.638049 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:07:59.638049 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:07:59.638049 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:07:59.638049 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:07:59.638049 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:07:59.638049 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:07:59.638049 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:07:59.638049 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 581/1180] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 05:07:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:07:59.643920 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875240b-a6bf-4ef4-b768-c0d262a3bc46 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:59.731947 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:59.748142 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875240b-a6bf-4ef4-b768-c0d262a3bc46 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:59.781813 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2328eb28-59f5-4bcd-9d94-7689410bf258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:59.782441 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328eb28-59f5-4bcd-9d94-7689410bf258 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:59.782716 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328eb28-59f5-4bcd-9d94-7689410bf258 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:59.782980 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328eb28-59f5-4bcd-9d94-7689410bf258 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:59.783637 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2328eb28-59f5-4bcd-9d94-7689410bf258 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:07:59.783940 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2328eb28-59f5-4bcd-9d94-7689410bf258 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:59.785304 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2328eb28-59f5-4bcd-9d94-7689410bf258 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:59.795631 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 600/1181] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:07:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:59.802567 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7551a6-1960-452c-b9c0-59b66d0f6db2 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:07:59.910783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7551a6-1960-452c-b9c0-59b66d0f6db2 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:07:59.949476 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ba2e9-a903-4b5a-ab23-c2aec6e08b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:07:59.950128 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ba2e9-a903-4b5a-ab23-c2aec6e08b75 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:07:59.950413 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ba2e9-a903-4b5a-ab23-c2aec6e08b75 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:07:59.950677 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ba2e9-a903-4b5a-ab23-c2aec6e08b75 None admin] PATH_INFO: `/v3/users/466bbb2101684438b2c83110744d1ad9` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:07:59.952853 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:07:59.964973 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50ba2e9-a903-4b5a-ab23-c2aec6e08b75 None admin] RBAC: Authorizing `identity:get_user(user_id=466bbb2101684438b2c83110744d1ad9)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:07:59.966505 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50ba2e9-a903-4b5a-ab23-c2aec6e08b75 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:07:59.968118 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 582/1182] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:07:59 2022] GET /identity/v3/users/466bbb2101684438b2c83110744d1ad9 => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:07:59.974552 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2328eb28-59f5-4bcd-9d94-7689410bf258 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:08:00.081410 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2328eb28-59f5-4bcd-9d94-7689410bf258 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:08:00.119427 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468149cc-37dd-46bc-a0de-529763137c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:08:00.120053 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468149cc-37dd-46bc-a0de-529763137c31 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:08:00.120262 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468149cc-37dd-46bc-a0de-529763137c31 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:08:00.120525 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468149cc-37dd-46bc-a0de-529763137c31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:08:00.138849 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:08:00.192815 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468149cc-37dd-46bc-a0de-529763137c31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:08:00.194263 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468149cc-37dd-46bc-a0de-529763137c31 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:08:00.296014 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 601/1183] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:07:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 05:08:00.305900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ba2e9-a903-4b5a-ab23-c2aec6e08b75 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:08:00.420852 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ba2e9-a903-4b5a-ab23-c2aec6e08b75 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:08:00.456127 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe095ec-92bf-41bc-8b3d-076b71175b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:08:00.456759 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe095ec-92bf-41bc-8b3d-076b71175b71 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:08:00.457041 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe095ec-92bf-41bc-8b3d-076b71175b71 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:08:00.457308 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe095ec-92bf-41bc-8b3d-076b71175b71 None admin] PATH_INFO: `/v3/projects/4270165de51a4684a14cdc180da02444` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:08:00.458496 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe095ec-92bf-41bc-8b3d-076b71175b71 None admin] RBAC: Authorizing `identity:get_project(project_id=4270165de51a4684a14cdc180da02444)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:08:00.459955 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe095ec-92bf-41bc-8b3d-076b71175b71 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:08:00.461634 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 583/1184] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:08:00 2022] GET /identity/v3/projects/4270165de51a4684a14cdc180da02444 => generated 361 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:08:00.468278 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468149cc-37dd-46bc-a0de-529763137c31 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:08:00.520017 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:08:00.571830 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468149cc-37dd-46bc-a0de-529763137c31 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:08:00.609566 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cdbce1-013b-43b6-92b5-5e9e0c09680c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:08:00.610187 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdbce1-013b-43b6-92b5-5e9e0c09680c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:08:00.610471 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdbce1-013b-43b6-92b5-5e9e0c09680c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:08:00.610736 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdbce1-013b-43b6-92b5-5e9e0c09680c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:08:00.611467 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cdbce1-013b-43b6-92b5-5e9e0c09680c None admin] RBAC: Adding query filter params (role.id=c3d5589dd2864be9a0f26f5802d874cd, scope.project.id=4270165de51a4684a14cdc180da02444, user.id=466bbb2101684438b2c83110744d1ad9) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:08:00.611765 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cdbce1-013b-43b6-92b5-5e9e0c09680c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:08:00.613220 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cdbce1-013b-43b6-92b5-5e9e0c09680c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:08:00.619522 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 602/1185] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:08:00 2022] GET /identity/v3/role_assignments?user.id=466bbb2101684438b2c83110744d1ad9&scope.project.id=4270165de51a4684a14cdc180da02444&role.id=c3d5589dd2864be9a0f26f5802d874cd => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:08:02.458642 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f976b011-5d31-49eb-a4ff-1d52cb865a38 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:08:02.459219 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f976b011-5d31-49eb-a4ff-1d52cb865a38 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:08:02.459219 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f976b011-5d31-49eb-a4ff-1d52cb865a38 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:08:02.461216 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 584/1186] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:08:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:08:02.469888 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06889230-3da9-40c6-956e-58cce5b54c1e None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:08:02.470326 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06889230-3da9-40c6-956e-58cce5b54c1e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:08:02.470517 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06889230-3da9-40c6-956e-58cce5b54c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:08:02.493495 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-06889230-3da9-40c6-956e-58cce5b54c1e None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:08:02.494738 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-06889230-3da9-40c6-956e-58cce5b54c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:08:02.563667 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 603/1187] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:08:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:08:02.835484 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f976b011-5d31-49eb-a4ff-1d52cb865a38 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:08:02.902661 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:08:02.947325 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f976b011-5d31-49eb-a4ff-1d52cb865a38 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:08:02.982052 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6641ee-c988-43f8-95c9-476355170ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:08:02.982688 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6641ee-c988-43f8-95c9-476355170ebc None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:08:02.982962 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6641ee-c988-43f8-95c9-476355170ebc None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:08:02.983226 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6641ee-c988-43f8-95c9-476355170ebc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:08:02.993938 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6641ee-c988-43f8-95c9-476355170ebc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:08:02.998666 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-8f6641ee-c988-43f8-95c9-476355170ebc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:08:02.998666 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:08:02.998666 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:08:02.998666 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:08:02.998666 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:08:02.998666 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:08:02.998666 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:08:02.998666 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:08:02.998666 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:08:02.998666 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:08:02.998666 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:08:02.998666 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:08:02.998666 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:08:02.998666 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:08:02.998666 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:08:02.998666 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:08:02.999774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:08:02.999774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:08:02.999774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:08:02.999774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:08:02.999774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:08:02.999774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:08:02.999774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:08:02.999774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:08:02.999774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:08:02.999774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:08:02.999774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:08:02.999774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:08:02.999774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:08:02.999774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:08:02.999774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:08:02.999774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:08:03.000712 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:08:03.000712 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:08:03.000712 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:08:03.000712 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:08:03.000712 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:08:03.000712 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:08:03.000712 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:08:03.000712 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:08:03.000712 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:08:03.000712 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:08:03.000712 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:08:03.000712 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:08:03.000712 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 585/1188] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 05:08:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:08:03.008063 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06889230-3da9-40c6-956e-58cce5b54c1e None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:08:03.052632 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:08:03.111788 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06889230-3da9-40c6-956e-58cce5b54c1e None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:08:03.146877 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2f20f9-4a5e-450a-a95f-0d63b8cf1734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:08:03.147465 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f20f9-4a5e-450a-a95f-0d63b8cf1734 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:08:03.147737 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f20f9-4a5e-450a-a95f-0d63b8cf1734 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:08:03.148000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f20f9-4a5e-450a-a95f-0d63b8cf1734 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:08:03.148646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2f20f9-4a5e-450a-a95f-0d63b8cf1734 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:08:03.148939 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2f20f9-4a5e-450a-a95f-0d63b8cf1734 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:08:03.150322 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2f20f9-4a5e-450a-a95f-0d63b8cf1734 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:08:03.158505 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 604/1189] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:08:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:08:03.164927 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6641ee-c988-43f8-95c9-476355170ebc None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:08:03.279071 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6641ee-c988-43f8-95c9-476355170ebc None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:08:03.300425 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:08:03.317452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8828d568-8e04-4806-822c-21f12ff13b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:08:03.318065 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828d568-8e04-4806-822c-21f12ff13b09 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:08:03.318364 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828d568-8e04-4806-822c-21f12ff13b09 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:08:03.318628 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828d568-8e04-4806-822c-21f12ff13b09 None admin] PATH_INFO: `/v3/users/466bbb2101684438b2c83110744d1ad9` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:08:03.319867 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8828d568-8e04-4806-822c-21f12ff13b09 None admin] RBAC: Authorizing `identity:get_user(user_id=466bbb2101684438b2c83110744d1ad9)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:08:03.321310 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8828d568-8e04-4806-822c-21f12ff13b09 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:08:03.322975 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 586/1190] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:08:03 2022] GET /identity/v3/users/466bbb2101684438b2c83110744d1ad9 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:08:03.328854 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2f20f9-4a5e-450a-a95f-0d63b8cf1734 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:08:03.430996 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2f20f9-4a5e-450a-a95f-0d63b8cf1734 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:08:03.433328 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:08:03.465867 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceba9bb-6ba0-45c3-95f7-f88597623378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:08:03.466530 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceba9bb-6ba0-45c3-95f7-f88597623378 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:08:03.466801 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceba9bb-6ba0-45c3-95f7-f88597623378 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:08:03.467068 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceba9bb-6ba0-45c3-95f7-f88597623378 None admin] PATH_INFO: `/v3/projects/4270165de51a4684a14cdc180da02444` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:08:03.468187 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceba9bb-6ba0-45c3-95f7-f88597623378 None admin] RBAC: Authorizing `identity:get_project(project_id=4270165de51a4684a14cdc180da02444)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:08:03.469607 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceba9bb-6ba0-45c3-95f7-f88597623378 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:08:03.471237 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 605/1191] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:08:03 2022] GET /identity/v3/projects/4270165de51a4684a14cdc180da02444 => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:08:03.477657 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8828d568-8e04-4806-822c-21f12ff13b09 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:08:03.595865 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8828d568-8e04-4806-822c-21f12ff13b09 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:08:03.634313 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98e6966-83fa-4a17-b488-4b717f988ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:08:03.634954 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98e6966-83fa-4a17-b488-4b717f988ab3 None admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:08:03.635254 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98e6966-83fa-4a17-b488-4b717f988ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:08:03.635522 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98e6966-83fa-4a17-b488-4b717f988ab3 None admin] PATH_INFO: `/v3/projects/4270165de51a4684a14cdc180da02444/users/466bbb2101684438b2c83110744d1ad9/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:08:03.637648 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98e6966-83fa-4a17-b488-4b717f988ab3 None admin] RBAC: Authorizing `identity:create_grant(project_id=4270165de51a4684a14cdc180da02444, user_id=466bbb2101684438b2c83110744d1ad9, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:08:03.639121 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98e6966-83fa-4a17-b488-4b717f988ab3 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:08:03.658450 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 587/1192] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 05:08:03 2022] PUT /identity/v3/projects/4270165de51a4684a14cdc180da02444/users/466bbb2101684438b2c83110744d1ad9/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:08:05.505282 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06684d7-5c38-4b95-9f3f-8a13b7f55357 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:08:05.505842 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06684d7-5c38-4b95-9f3f-8a13b7f55357 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:08:05.505842 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06684d7-5c38-4b95-9f3f-8a13b7f55357 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:08:05.507178 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 606/1193] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:08:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:08:05.513642 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc84fc-1247-4fe7-a2e9-022e153cc0ba None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:08:05.514411 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc84fc-1247-4fe7-a2e9-022e153cc0ba None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:08:05.514570 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc84fc-1247-4fe7-a2e9-022e153cc0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:08:05.544116 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-29dc84fc-1247-4fe7-a2e9-022e153cc0ba None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:08:05.545321 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-29dc84fc-1247-4fe7-a2e9-022e153cc0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:08:05.610204 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:08:05.636022 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 588/1194] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:08:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:08:05.899040 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06684d7-5c38-4b95-9f3f-8a13b7f55357 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:08:06.015774 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06684d7-5c38-4b95-9f3f-8a13b7f55357 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:08:06.050099 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0019b74-83b2-4a59-85d9-aa311c3617c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:08:06.050718 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0019b74-83b2-4a59-85d9-aa311c3617c5 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:08:06.050991 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0019b74-83b2-4a59-85d9-aa311c3617c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:08:06.051258 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0019b74-83b2-4a59-85d9-aa311c3617c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:08:06.062749 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0019b74-83b2-4a59-85d9-aa311c3617c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:08:06.067475 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-e0019b74-83b2-4a59-85d9-aa311c3617c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:08:06.067475 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:08:06.067475 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:08:06.067475 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:08:06.067475 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:08:06.067475 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:08:06.067475 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:08:06.067475 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:08:06.067475 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:08:06.067475 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:08:06.067475 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:08:06.067475 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:08:06.067475 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:08:06.067475 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:08:06.067475 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:08:06.067475 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:08:06.069092 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:08:06.069092 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:08:06.069092 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:08:06.069092 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:08:06.069092 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:08:06.069092 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:08:06.069092 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:08:06.069092 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:08:06.069092 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:08:06.069092 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:08:06.069092 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:08:06.069092 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:08:06.069092 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:08:06.069092 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:08:06.069092 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:08:06.069092 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:08:06.070251 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:08:06.070251 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:08:06.070251 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:08:06.070251 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:08:06.070251 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:08:06.070251 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:08:06.070251 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:08:06.070251 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:08:06.070251 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:08:06.070251 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:08:06.070251 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:08:06.070251 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:08:06.070251 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 607/1195] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 05:08:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:08:06.075468 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc84fc-1247-4fe7-a2e9-022e153cc0ba None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:08:06.177273 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc84fc-1247-4fe7-a2e9-022e153cc0ba None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:08:06.212795 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045016e-9891-411f-a22b-e3c943a3013b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:08:06.213389 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045016e-9891-411f-a22b-e3c943a3013b None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:08:06.213667 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045016e-9891-411f-a22b-e3c943a3013b None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:08:06.213934 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045016e-9891-411f-a22b-e3c943a3013b None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:08:06.214783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8045016e-9891-411f-a22b-e3c943a3013b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:08:06.215091 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8045016e-9891-411f-a22b-e3c943a3013b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:08:06.216461 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8045016e-9891-411f-a22b-e3c943a3013b None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:08:06.224879 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 589/1196] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:08:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:08:06.231101 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0019b74-83b2-4a59-85d9-aa311c3617c5 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:08:06.265824 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:08:06.340791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0019b74-83b2-4a59-85d9-aa311c3617c5 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:08:06.387498 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfef91a-1902-47db-a3cf-cda404f73257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:08:06.388435 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfef91a-1902-47db-a3cf-cda404f73257 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:08:06.388796 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfef91a-1902-47db-a3cf-cda404f73257 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:08:06.389124 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfef91a-1902-47db-a3cf-cda404f73257 None admin] PATH_INFO: `/v3/users/466bbb2101684438b2c83110744d1ad9` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:08:06.391068 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfef91a-1902-47db-a3cf-cda404f73257 None admin] RBAC: Authorizing `identity:get_user(user_id=466bbb2101684438b2c83110744d1ad9)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:08:06.393322 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfef91a-1902-47db-a3cf-cda404f73257 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:08:06.395836 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 608/1197] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Aug 31 05:08:06 2022] GET /identity/v3/users/466bbb2101684438b2c83110744d1ad9 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:08:06.402365 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045016e-9891-411f-a22b-e3c943a3013b None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:08:06.445278 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:08:06.508905 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045016e-9891-411f-a22b-e3c943a3013b None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:08:06.542233 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af9732-1486-4b20-9f0b-40b4eeb9fd43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:08:06.542921 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af9732-1486-4b20-9f0b-40b4eeb9fd43 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:08:06.543198 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af9732-1486-4b20-9f0b-40b4eeb9fd43 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:08:06.543466 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af9732-1486-4b20-9f0b-40b4eeb9fd43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:08:06.614375 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89af9732-1486-4b20-9f0b-40b4eeb9fd43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:08:06.616171 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89af9732-1486-4b20-9f0b-40b4eeb9fd43 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:08:06.657546 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:08:06.717342 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 590/1198] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:08:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 05:08:06.724021 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfef91a-1902-47db-a3cf-cda404f73257 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:08:06.824290 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:08:06.827472 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfef91a-1902-47db-a3cf-cda404f73257 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:08:06.864838 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a9ebf6-0435-4e32-801d-969445f9654e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:08:06.865441 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a9ebf6-0435-4e32-801d-969445f9654e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:08:06.865711 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a9ebf6-0435-4e32-801d-969445f9654e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:08:06.866332 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a9ebf6-0435-4e32-801d-969445f9654e None admin] PATH_INFO: `/v3/projects/4270165de51a4684a14cdc180da02444` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:08:06.867511 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a9ebf6-0435-4e32-801d-969445f9654e None admin] RBAC: Authorizing `identity:get_project(project_id=4270165de51a4684a14cdc180da02444)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:08:06.868936 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a9ebf6-0435-4e32-801d-969445f9654e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:08:06.870600 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 609/1199] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Aug 31 05:08:06 2022] GET /identity/v3/projects/4270165de51a4684a14cdc180da02444 => generated 361 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:08:06.876854 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af9732-1486-4b20-9f0b-40b4eeb9fd43 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:08:06.978555 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af9732-1486-4b20-9f0b-40b4eeb9fd43 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:08:07.013875 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5885c10f-568b-42ad-b12d-f4eb65558b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:08:07.014489 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885c10f-568b-42ad-b12d-f4eb65558b1c None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:08:07.014761 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885c10f-568b-42ad-b12d-f4eb65558b1c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:08:07.015023 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885c10f-568b-42ad-b12d-f4eb65558b1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:08:07.015744 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5885c10f-568b-42ad-b12d-f4eb65558b1c None admin] RBAC: Adding query filter params (role.id=c3d5589dd2864be9a0f26f5802d874cd, scope.project.id=4270165de51a4684a14cdc180da02444, user.id=466bbb2101684438b2c83110744d1ad9) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:08:07.016042 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5885c10f-568b-42ad-b12d-f4eb65558b1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:08:07.017477 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5885c10f-568b-42ad-b12d-f4eb65558b1c None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:08:07.023473 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 591/1200] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:08:06 2022] GET /identity/v3/role_assignments?user.id=466bbb2101684438b2c83110744d1ad9&scope.project.id=4270165de51a4684a14cdc180da02444&role.id=c3d5589dd2864be9a0f26f5802d874cd => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:08:16.860199 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=16 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:08:17.020780 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=98 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:01.870428 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb7650f-70fd-45b3-beec-3cebb10c11f2 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:01.871050 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb7650f-70fd-45b3-beec-3cebb10c11f2 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:01.871050 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb7650f-70fd-45b3-beec-3cebb10c11f2 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:01.872517 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 610/1201] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:09:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:09:01.878596 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898e36a-9a8f-4e10-979a-cbd23b8fd2df None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:01.879104 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898e36a-9a8f-4e10-979a-cbd23b8fd2df None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:01.879104 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898e36a-9a8f-4e10-979a-cbd23b8fd2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:01.898567 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-1898e36a-9a8f-4e10-979a-cbd23b8fd2df None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:09:01.899811 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-1898e36a-9a8f-4e10-979a-cbd23b8fd2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:09:01.983647 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 592/1202] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:09:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:09:02.259591 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb7650f-70fd-45b3-beec-3cebb10c11f2 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:02.397698 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb7650f-70fd-45b3-beec-3cebb10c11f2 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:02.436794 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26737c99-5c3a-4872-a373-ec5a75b9a29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:02.437496 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26737c99-5c3a-4872-a373-ec5a75b9a29a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:02.437835 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26737c99-5c3a-4872-a373-ec5a75b9a29a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:02.438230 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26737c99-5c3a-4872-a373-ec5a75b9a29a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:02.454198 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26737c99-5c3a-4872-a373-ec5a75b9a29a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:02.456020 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26737c99-5c3a-4872-a373-ec5a75b9a29a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:02.473745 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-26737c99-5c3a-4872-a373-ec5a75b9a29a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:09:02.473745 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:09:02.473745 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:09:02.473745 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:09:02.473745 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:09:02.473745 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:09:02.473745 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:09:02.473745 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:09:02.473745 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:09:02.473745 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:09:02.473745 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:09:02.473745 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:09:02.473745 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:09:02.473745 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:09:02.473745 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:09:02.473745 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:09:02.475079 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:09:02.475079 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:09:02.475079 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:09:02.475079 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:09:02.475079 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:09:02.475079 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:09:02.475079 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:09:02.475079 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:09:02.475079 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:09:02.475079 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:09:02.475079 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:09:02.475079 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:09:02.475079 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:09:02.475079 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:09:02.475079 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:09:02.475079 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:09:02.475079 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:09:02.476323 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:09:02.476323 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:09:02.476323 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:09:02.476323 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 611/1203] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:09:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:09:02.481698 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898e36a-9a8f-4e10-979a-cbd23b8fd2df None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:02.603936 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898e36a-9a8f-4e10-979a-cbd23b8fd2df None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:02.645173 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:02.653170 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc8e1c-ff2a-40ff-8d73-6e477740059d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:02.653822 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc8e1c-ff2a-40ff-8d73-6e477740059d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:02.654232 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc8e1c-ff2a-40ff-8d73-6e477740059d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:02.654601 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc8e1c-ff2a-40ff-8d73-6e477740059d None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:02.655358 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dc8e1c-ff2a-40ff-8d73-6e477740059d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:09:02.655769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dc8e1c-ff2a-40ff-8d73-6e477740059d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:02.657604 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dc8e1c-ff2a-40ff-8d73-6e477740059d None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:02.676199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 593/1204] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:09:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:09:02.682645 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26737c99-5c3a-4872-a373-ec5a75b9a29a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:02.748414 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:02.815177 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26737c99-5c3a-4872-a373-ec5a75b9a29a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:02.855001 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00c1588-3a2f-4871-8714-fd6678539a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:02.855570 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c1588-3a2f-4871-8714-fd6678539a1f None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:02.855859 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c1588-3a2f-4871-8714-fd6678539a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:02.856240 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c1588-3a2f-4871-8714-fd6678539a1f None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:02.857095 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00c1588-3a2f-4871-8714-fd6678539a1f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:02.858909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00c1588-3a2f-4871-8714-fd6678539a1f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:02.890498 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 612/1205] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Aug 31 05:09:02 2022] POST /identity/v3/users => generated 251 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:09:04.837107 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab0f06-0cdb-41de-8ce2-03c3b5f469b4 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:04.837712 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab0f06-0cdb-41de-8ce2-03c3b5f469b4 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:04.837712 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab0f06-0cdb-41de-8ce2-03c3b5f469b4 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:04.839200 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 594/1206] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:09:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 05:09:04.844372 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d46d0-8635-4a34-ac68-8907a0f588df None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:04.844811 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d46d0-8635-4a34-ac68-8907a0f588df None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:04.844909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d46d0-8635-4a34-ac68-8907a0f588df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:04.865914 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-6e3d46d0-8635-4a34-ac68-8907a0f588df None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:09:04.867196 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6e3d46d0-8635-4a34-ac68-8907a0f588df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:09:04.941788 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 613/1207] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:09:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:09:05.203268 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab0f06-0cdb-41de-8ce2-03c3b5f469b4 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:05.321759 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab0f06-0cdb-41de-8ce2-03c3b5f469b4 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:05.365491 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68ebe0-dc28-4ce4-b97b-822074e2702c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:05.366558 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68ebe0-dc28-4ce4-b97b-822074e2702c None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:05.367095 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68ebe0-dc28-4ce4-b97b-822074e2702c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:05.367561 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68ebe0-dc28-4ce4-b97b-822074e2702c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:05.380373 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee68ebe0-dc28-4ce4-b97b-822074e2702c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:09:05.385731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-ee68ebe0-dc28-4ce4-b97b-822074e2702c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 05:09:05.385731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:09:05.385731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:09:05.385731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:09:05.385731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:09:05.385731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:09:05.385731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:09:05.385731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:09:05.385731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:09:05.385731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:09:05.385731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:09:05.385731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:09:05.385731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:09:05.385731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:09:05.385731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:09:05.385731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:09:05.386792 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:09:05.386792 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:09:05.386792 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:09:05.386792 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:09:05.386792 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:09:05.386792 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:09:05.386792 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:09:05.386792 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:09:05.386792 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:09:05.386792 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:09:05.386792 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:09:05.386792 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:09:05.386792 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:09:05.386792 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:09:05.386792 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:09:05.386792 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:09:05.388169 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:09:05.388169 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:09:05.388169 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:09:05.388169 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:09:05.388169 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:09:05.388169 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:09:05.388169 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:09:05.388169 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:09:05.388169 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:09:05.388169 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:09:05.388169 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 05:09:05.388169 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:09:05.388169 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 595/1208] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 05:09:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:09:05.392930 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3d46d0-8635-4a34-ac68-8907a0f588df None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:05.436399 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:05.503117 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3d46d0-8635-4a34-ac68-8907a0f588df None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:05.540847 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d552f612-2d82-4fab-be42-d825f7b5a6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:05.541486 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552f612-2d82-4fab-be42-d825f7b5a6ad None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:05.541758 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552f612-2d82-4fab-be42-d825f7b5a6ad None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:05.542046 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552f612-2d82-4fab-be42-d825f7b5a6ad None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:05.542716 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d552f612-2d82-4fab-be42-d825f7b5a6ad None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:09:05.543130 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d552f612-2d82-4fab-be42-d825f7b5a6ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:05.544482 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d552f612-2d82-4fab-be42-d825f7b5a6ad None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:05.553160 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 614/1209] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Aug 31 05:09:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:09:05.559039 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68ebe0-dc28-4ce4-b97b-822074e2702c None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:05.629687 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:05.696855 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68ebe0-dc28-4ce4-b97b-822074e2702c None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:05.741502 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d82786-ee8b-4c88-b09a-58a954d8df0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:05.742218 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d82786-ee8b-4c88-b09a-58a954d8df0e None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:05.742483 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d82786-ee8b-4c88-b09a-58a954d8df0e None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:05.742777 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d82786-ee8b-4c88-b09a-58a954d8df0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:05.753721 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d82786-ee8b-4c88-b09a-58a954d8df0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:05.755242 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d82786-ee8b-4c88-b09a-58a954d8df0e None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:05.770878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-45d82786-ee8b-4c88-b09a-58a954d8df0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:09:05.770878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:09:05.770878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:09:05.770878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:09:05.770878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:09:05.770878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:09:05.770878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:09:05.770878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:09:05.770878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:09:05.770878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:09:05.770878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:09:05.770878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:09:05.770878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:09:05.770878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:09:05.770878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:09:05.770878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:09:05.771894 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:09:05.771894 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:09:05.771894 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:09:05.771894 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:09:05.771894 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:09:05.771894 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:09:05.771894 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:09:05.771894 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:09:05.771894 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:09:05.771894 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:09:05.771894 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:09:05.771894 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:09:05.771894 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:09:05.771894 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:09:05.771894 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:09:05.771894 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:09:05.771894 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:09:05.772807 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:09:05.772807 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:09:05.772807 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:09:05.772807 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 596/1210] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:09:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:09:05.778146 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d552f612-2d82-4fab-be42-d825f7b5a6ad None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:05.896117 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:05.900372 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d552f612-2d82-4fab-be42-d825f7b5a6ad None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:05.941991 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b99ef-64e6-4db6-933a-8fed18c76f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:05.942767 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b99ef-64e6-4db6-933a-8fed18c76f0a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:05.943082 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b99ef-64e6-4db6-933a-8fed18c76f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:05.943413 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b99ef-64e6-4db6-933a-8fed18c76f0a None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:05.944165 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903b99ef-64e6-4db6-933a-8fed18c76f0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:09:05.944470 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903b99ef-64e6-4db6-933a-8fed18c76f0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:05.946217 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903b99ef-64e6-4db6-933a-8fed18c76f0a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:05.963580 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 615/1211] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:09:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:09:05.970396 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d82786-ee8b-4c88-b09a-58a954d8df0e None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:06.092182 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d82786-ee8b-4c88-b09a-58a954d8df0e None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:06.094105 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:06.136509 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bf9718-3b7b-4a75-adeb-b4f337869525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:06.137340 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf9718-3b7b-4a75-adeb-b4f337869525 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:06.137744 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf9718-3b7b-4a75-adeb-b4f337869525 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:06.138189 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf9718-3b7b-4a75-adeb-b4f337869525 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:06.148403 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bf9718-3b7b-4a75-adeb-b4f337869525 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:06.150533 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bf9718-3b7b-4a75-adeb-b4f337869525 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:06.164907 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-e6bf9718-3b7b-4a75-adeb-b4f337869525 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 05:09:06.164907 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:09:06.164907 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:09:06.164907 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:09:06.164907 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:09:06.164907 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:09:06.164907 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:09:06.164907 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:09:06.164907 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:09:06.164907 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:09:06.164907 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:09:06.164907 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:09:06.164907 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 05:09:06.164907 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 05:09:06.164907 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 05:09:06.164907 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 05:09:06.166167 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:09:06.166167 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:09:06.166167 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 05:09:06.166167 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:09:06.166167 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 05:09:06.166167 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:09:06.166167 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:09:06.166167 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:09:06.166167 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:09:06.166167 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:09:06.166167 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:09:06.166167 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:09:06.166167 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:09:06.166167 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:09:06.166167 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:09:06.166167 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:09:06.166167 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:09:06.166167 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:09:06.167163 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:09:06.167163 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:09:06.167163 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 05:09:06.167163 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 05:09:06.167163 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 05:09:06.167163 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 05:09:06.167163 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 05:09:06.167163 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 05:09:06.167163 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 05:09:06.167163 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:09:06.167163 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 597/1212] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:09:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:09:06.172531 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b99ef-64e6-4db6-933a-8fed18c76f0a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:06.300294 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b99ef-64e6-4db6-933a-8fed18c76f0a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:06.344120 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9036c4-017b-4574-bb82-3fe8aca68d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:06.344865 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9036c4-017b-4574-bb82-3fe8aca68d4d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:06.345199 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9036c4-017b-4574-bb82-3fe8aca68d4d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:06.345551 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9036c4-017b-4574-bb82-3fe8aca68d4d None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:06.346668 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9036c4-017b-4574-bb82-3fe8aca68d4d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:09:06.347016 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9036c4-017b-4574-bb82-3fe8aca68d4d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:06.348961 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9036c4-017b-4574-bb82-3fe8aca68d4d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:06.376966 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 616/1213] 10.222.0.36 () {56 vars in 1143 bytes} [Wed Aug 31 05:09:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:09:06.378901 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bf9718-3b7b-4a75-adeb-b4f337869525 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:06.516041 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bf9718-3b7b-4a75-adeb-b4f337869525 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:06.554341 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7cadc6-e02e-4b12-998e-3200deda370a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:06.554996 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7cadc6-e02e-4b12-998e-3200deda370a None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:06.555278 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7cadc6-e02e-4b12-998e-3200deda370a None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:06.555539 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7cadc6-e02e-4b12-998e-3200deda370a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:06.588522 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:06.641112 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7cadc6-e02e-4b12-998e-3200deda370a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:06.642588 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7cadc6-e02e-4b12-998e-3200deda370a None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:06.770982 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 598/1214] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:09:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 05:09:06.777418 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9036c4-017b-4574-bb82-3fe8aca68d4d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:06.806966 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:06.913461 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9036c4-017b-4574-bb82-3fe8aca68d4d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:06.960185 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e14dde-e327-4814-ba50-3330554cbc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:06.961337 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e14dde-e327-4814-ba50-3330554cbc79 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:06.961800 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e14dde-e327-4814-ba50-3330554cbc79 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:06.962353 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e14dde-e327-4814-ba50-3330554cbc79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:06.973110 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e14dde-e327-4814-ba50-3330554cbc79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:06.974651 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e14dde-e327-4814-ba50-3330554cbc79 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:06.989378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-36e14dde-e327-4814-ba50-3330554cbc79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:09:06.989378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:09:06.989378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:09:06.989378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:09:06.989378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:09:06.989378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:09:06.989378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:09:06.989378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:09:06.989378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:09:06.989378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:09:06.989378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:09:06.989378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:09:06.989378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:09:06.989378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:09:06.989378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:09:06.989378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:09:06.990696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:09:06.990696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:09:06.990696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:09:06.990696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:09:06.990696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:09:06.990696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:09:06.990696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:09:06.990696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:09:06.990696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:09:06.990696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:09:06.990696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:09:06.990696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:09:06.990696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:09:06.990696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:09:06.990696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:09:06.990696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:09:06.990696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:09:06.995090 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:09:06.995090 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:09:06.995090 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:09:06.995090 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 617/1215] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:09:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:09:06.996627 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7cadc6-e02e-4b12-998e-3200deda370a None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:07.057679 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:07.099720 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7cadc6-e02e-4b12-998e-3200deda370a None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:07.134960 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bda2a8-1346-4961-938c-3b6a576d6e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:07.135677 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bda2a8-1346-4961-938c-3b6a576d6e8f None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:07.136015 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bda2a8-1346-4961-938c-3b6a576d6e8f None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:07.136332 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bda2a8-1346-4961-938c-3b6a576d6e8f None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:07.137056 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bda2a8-1346-4961-938c-3b6a576d6e8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:09:07.137375 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bda2a8-1346-4961-938c-3b6a576d6e8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:07.138903 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bda2a8-1346-4961-938c-3b6a576d6e8f None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:07.152418 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 599/1216] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:09:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:09:07.158798 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e14dde-e327-4814-ba50-3330554cbc79 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:07.243643 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:07.276866 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e14dde-e327-4814-ba50-3330554cbc79 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:07.313293 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573bbbbb-0793-4662-9d46-74376c584d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:07.313948 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573bbbbb-0793-4662-9d46-74376c584d10 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:07.314253 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573bbbbb-0793-4662-9d46-74376c584d10 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:07.314630 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573bbbbb-0793-4662-9d46-74376c584d10 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:07.322794 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573bbbbb-0793-4662-9d46-74376c584d10 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:07.324492 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573bbbbb-0793-4662-9d46-74376c584d10 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:07.337527 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-573bbbbb-0793-4662-9d46-74376c584d10 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:09:07.337527 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:09:07.337527 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:09:07.337527 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:09:07.337527 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:09:07.337527 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:09:07.337527 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:09:07.337527 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:09:07.337527 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:09:07.337527 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:09:07.337527 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:09:07.337527 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:09:07.337527 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 05:09:07.337527 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 05:09:07.337527 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 05:09:07.338857 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 05:09:07.338857 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:09:07.338857 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:09:07.338857 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:09:07.338857 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:09:07.338857 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:09:07.338857 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:09:07.338857 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:09:07.338857 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:09:07.338857 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:09:07.338857 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:09:07.338857 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:09:07.338857 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:09:07.338857 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:09:07.338857 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:09:07.338857 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:09:07.338857 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:09:07.339868 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 05:09:07.339868 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 05:09:07.339868 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 05:09:07.339868 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 05:09:07.339868 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 05:09:07.339868 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 05:09:07.339868 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:09:07.339868 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:09:07.339868 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 618/1217] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:09:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:09:07.348152 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bda2a8-1346-4961-938c-3b6a576d6e8f None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:07.468550 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bda2a8-1346-4961-938c-3b6a576d6e8f None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:07.494084 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:07.507175 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1e204b-f986-442b-bcdd-c380a4d1cf98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:07.507969 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e204b-f986-442b-bcdd-c380a4d1cf98 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:07.508212 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e204b-f986-442b-bcdd-c380a4d1cf98 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:07.508515 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e204b-f986-442b-bcdd-c380a4d1cf98 None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:07.509269 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1e204b-f986-442b-bcdd-c380a4d1cf98 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:09:07.509636 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1e204b-f986-442b-bcdd-c380a4d1cf98 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:07.511286 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1e204b-f986-442b-bcdd-c380a4d1cf98 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:07.525208 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 600/1218] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 05:09:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:09:07.532325 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573bbbbb-0793-4662-9d46-74376c584d10 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:07.656451 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573bbbbb-0793-4662-9d46-74376c584d10 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:07.691717 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:07.698842 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e08b6-e5ab-4272-91f4-221e01ca08d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:07.699449 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e08b6-e5ab-4272-91f4-221e01ca08d7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:07.699722 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e08b6-e5ab-4272-91f4-221e01ca08d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:07.699984 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e08b6-e5ab-4272-91f4-221e01ca08d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:07.700756 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77e08b6-e5ab-4272-91f4-221e01ca08d7 None admin] RBAC: Adding query filter params (role.id=207bf98e31774cc0b21249de6d96155b, scope.project.id=e01ce1b1282e44a39d659b87f120995e, user.id=bd4387035099416789009f0ecb8ad282) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:09:07.701086 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77e08b6-e5ab-4272-91f4-221e01ca08d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:07.703239 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77e08b6-e5ab-4272-91f4-221e01ca08d7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:07.716326 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 619/1219] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:09:07 2022] GET /identity/v3/role_assignments?user.id=bd4387035099416789009f0ecb8ad282&scope.project.id=e01ce1b1282e44a39d659b87f120995e&role.id=207bf98e31774cc0b21249de6d96155b => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:09:09.733669 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5470c1f-039a-4b17-bec9-28add3857c5a None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:09.733669 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5470c1f-039a-4b17-bec9-28add3857c5a None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:09.733669 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5470c1f-039a-4b17-bec9-28add3857c5a None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:09.735581 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 601/1220] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:09:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:09:09.741420 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffec09a-794f-495d-9a5e-57053cc606bb None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:09.742133 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffec09a-794f-495d-9a5e-57053cc606bb None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:09.742133 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffec09a-794f-495d-9a5e-57053cc606bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:09.763145 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-8ffec09a-794f-495d-9a5e-57053cc606bb None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:09:09.764529 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8ffec09a-794f-495d-9a5e-57053cc606bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:09:09.842917 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 620/1221] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:09:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:09:10.132024 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5470c1f-039a-4b17-bec9-28add3857c5a None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:10.239833 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5470c1f-039a-4b17-bec9-28add3857c5a None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:10.276144 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6255eb90-3e40-409e-800e-c110d0884af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:10.276772 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255eb90-3e40-409e-800e-c110d0884af1 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:10.277156 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255eb90-3e40-409e-800e-c110d0884af1 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:10.277426 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255eb90-3e40-409e-800e-c110d0884af1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:10.288411 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6255eb90-3e40-409e-800e-c110d0884af1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:09:10.293235 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-6255eb90-3e40-409e-800e-c110d0884af1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 05:09:10.293235 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:09:10.293235 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:09:10.293235 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:09:10.293235 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:09:10.293235 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:09:10.293235 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:09:10.293235 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:09:10.293235 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:09:10.293235 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:09:10.293235 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:09:10.293235 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:09:10.293235 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:09:10.293235 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:09:10.293235 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:09:10.293235 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:09:10.294362 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:09:10.294362 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:09:10.294362 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:09:10.294362 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:09:10.294362 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:09:10.294362 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:09:10.294362 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:09:10.294362 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:09:10.294362 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:09:10.294362 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:09:10.294362 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:09:10.294362 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:09:10.294362 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:09:10.294362 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:09:10.294362 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:09:10.294362 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:09:10.295621 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:09:10.295621 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:09:10.295621 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:09:10.295621 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:09:10.295621 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:09:10.295621 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:09:10.295621 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:09:10.295621 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:09:10.295621 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:09:10.295621 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:09:10.295621 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 05:09:10.295621 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:09:10.295621 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 602/1222] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 05:09:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:09:10.300477 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffec09a-794f-495d-9a5e-57053cc606bb None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:10.404985 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffec09a-794f-495d-9a5e-57053cc606bb None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:10.435919 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:10.450644 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0536c3-f3d5-43ea-9a00-229702a33805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:10.451283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0536c3-f3d5-43ea-9a00-229702a33805 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:10.451561 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0536c3-f3d5-43ea-9a00-229702a33805 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:10.451823 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0536c3-f3d5-43ea-9a00-229702a33805 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:10.452487 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0536c3-f3d5-43ea-9a00-229702a33805 None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:09:10.452793 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0536c3-f3d5-43ea-9a00-229702a33805 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:10.454224 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0536c3-f3d5-43ea-9a00-229702a33805 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:10.462933 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 621/1223] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Aug 31 05:09:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:09:10.468774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6255eb90-3e40-409e-800e-c110d0884af1 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:10.515663 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:10.586640 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6255eb90-3e40-409e-800e-c110d0884af1 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:10.626373 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f67ea8-3de3-4418-a45c-41a1e233c948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:10.627072 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f67ea8-3de3-4418-a45c-41a1e233c948 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:10.627373 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f67ea8-3de3-4418-a45c-41a1e233c948 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:10.627688 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f67ea8-3de3-4418-a45c-41a1e233c948 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:10.641432 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f67ea8-3de3-4418-a45c-41a1e233c948 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:10.643212 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f67ea8-3de3-4418-a45c-41a1e233c948 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:10.664957 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-48f67ea8-3de3-4418-a45c-41a1e233c948 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:09:10.664957 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:09:10.664957 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:09:10.664957 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:09:10.664957 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:09:10.664957 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:09:10.664957 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:09:10.664957 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:09:10.664957 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:09:10.664957 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:09:10.664957 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:09:10.664957 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:09:10.664957 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:09:10.664957 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:09:10.664957 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:09:10.664957 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:09:10.665958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:09:10.665958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:09:10.665958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:09:10.665958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:09:10.665958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:09:10.665958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:09:10.665958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:09:10.665958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:09:10.665958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:09:10.665958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:09:10.665958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:09:10.665958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:09:10.665958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:09:10.665958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:09:10.665958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:09:10.665958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:09:10.665958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:09:10.667472 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:09:10.667472 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:09:10.667472 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:09:10.667472 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 603/1224] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:09:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:09:10.672476 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0536c3-f3d5-43ea-9a00-229702a33805 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:10.778937 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0536c3-f3d5-43ea-9a00-229702a33805 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:10.817020 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac033b2-c239-43b7-bdd7-2a8b5459fce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:10.818098 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac033b2-c239-43b7-bdd7-2a8b5459fce2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:10.818402 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac033b2-c239-43b7-bdd7-2a8b5459fce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:10.818809 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac033b2-c239-43b7-bdd7-2a8b5459fce2 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:10.819814 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac033b2-c239-43b7-bdd7-2a8b5459fce2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:09:10.820302 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac033b2-c239-43b7-bdd7-2a8b5459fce2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:10.822334 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac033b2-c239-43b7-bdd7-2a8b5459fce2 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:10.837459 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 622/1225] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:09:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:09:10.843695 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f67ea8-3de3-4418-a45c-41a1e233c948 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:10.953658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:10.963810 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f67ea8-3de3-4418-a45c-41a1e233c948 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:11.010119 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc310cf8-3f8a-4c17-bfba-eee328439aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:11.010756 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc310cf8-3f8a-4c17-bfba-eee328439aa9 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:11.011033 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc310cf8-3f8a-4c17-bfba-eee328439aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:11.011304 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc310cf8-3f8a-4c17-bfba-eee328439aa9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:11.019325 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc310cf8-3f8a-4c17-bfba-eee328439aa9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:11.021015 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc310cf8-3f8a-4c17-bfba-eee328439aa9 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:11.037787 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-bc310cf8-3f8a-4c17-bfba-eee328439aa9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 05:09:11.037787 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:09:11.037787 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:09:11.037787 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:09:11.037787 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:09:11.037787 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:09:11.037787 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:09:11.037787 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:09:11.037787 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:09:11.037787 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:09:11.037787 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:09:11.037787 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:09:11.037787 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 05:09:11.037787 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 05:09:11.037787 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 05:09:11.037787 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 05:09:11.040213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:09:11.040213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:09:11.040213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 05:09:11.040213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:09:11.040213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 05:09:11.040213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:09:11.040213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:09:11.040213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:09:11.040213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:09:11.040213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:09:11.040213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:09:11.040213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:09:11.040213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:09:11.040213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:09:11.040213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:09:11.040213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:09:11.040213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:09:11.040213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:09:11.042178 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:09:11.042178 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:09:11.042178 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 05:09:11.042178 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 05:09:11.042178 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 05:09:11.042178 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 05:09:11.042178 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 05:09:11.042178 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 05:09:11.042178 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 05:09:11.042178 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:09:11.042178 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 604/1226] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:09:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:09:11.046527 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac033b2-c239-43b7-bdd7-2a8b5459fce2 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:11.101997 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:11.160409 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac033b2-c239-43b7-bdd7-2a8b5459fce2 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:11.195160 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874437e-5776-470e-b04b-8b1803474b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:11.195761 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874437e-5776-470e-b04b-8b1803474b7e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:11.196038 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874437e-5776-470e-b04b-8b1803474b7e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:11.196302 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874437e-5776-470e-b04b-8b1803474b7e None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:11.197019 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3874437e-5776-470e-b04b-8b1803474b7e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:09:11.197317 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3874437e-5776-470e-b04b-8b1803474b7e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:11.198781 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3874437e-5776-470e-b04b-8b1803474b7e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:11.218549 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 623/1227] 10.222.0.36 () {56 vars in 1143 bytes} [Wed Aug 31 05:09:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:09:11.225609 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc310cf8-3f8a-4c17-bfba-eee328439aa9 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:11.337366 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc310cf8-3f8a-4c17-bfba-eee328439aa9 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:11.375071 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f395c7-f58c-4d52-b133-9a5c047b9bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:11.375701 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f395c7-f58c-4d52-b133-9a5c047b9bea None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:11.375974 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f395c7-f58c-4d52-b133-9a5c047b9bea None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:11.376242 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f395c7-f58c-4d52-b133-9a5c047b9bea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:11.387538 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f395c7-f58c-4d52-b133-9a5c047b9bea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:11.389088 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f395c7-f58c-4d52-b133-9a5c047b9bea None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:11.405499 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-15f395c7-f58c-4d52-b133-9a5c047b9bea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:09:11.405499 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:09:11.405499 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:09:11.405499 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:09:11.405499 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:09:11.405499 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:09:11.405499 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:09:11.405499 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:09:11.405499 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:09:11.405499 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:09:11.405499 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:09:11.405499 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:09:11.405499 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:09:11.405499 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:09:11.405499 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:09:11.405499 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:09:11.406672 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:09:11.406672 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:09:11.406672 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:09:11.406672 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:09:11.406672 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:09:11.406672 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:09:11.406672 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:09:11.406672 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:09:11.406672 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:09:11.406672 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:09:11.406672 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:09:11.406672 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:09:11.406672 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:09:11.406672 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:09:11.406672 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:09:11.406672 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:09:11.406672 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:09:11.407745 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:09:11.407745 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:09:11.407745 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:09:11.407745 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 605/1228] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:09:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:09:11.415471 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874437e-5776-470e-b04b-8b1803474b7e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:11.529232 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874437e-5776-470e-b04b-8b1803474b7e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:11.531729 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:11.564040 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910fac9-2209-439a-bfaa-288768cb1353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:11.564639 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910fac9-2209-439a-bfaa-288768cb1353 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:11.564914 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910fac9-2209-439a-bfaa-288768cb1353 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:11.565179 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910fac9-2209-439a-bfaa-288768cb1353 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:11.565816 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b910fac9-2209-439a-bfaa-288768cb1353 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:09:11.566136 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b910fac9-2209-439a-bfaa-288768cb1353 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:11.567502 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b910fac9-2209-439a-bfaa-288768cb1353 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:11.579017 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 624/1229] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:09:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:09:11.585152 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f395c7-f58c-4d52-b133-9a5c047b9bea None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:11.595629 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:11.700859 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f395c7-f58c-4d52-b133-9a5c047b9bea None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:11.733580 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f03919-c2db-4e86-8e56-dde91889baba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:11.734247 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f03919-c2db-4e86-8e56-dde91889baba None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:11.734517 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f03919-c2db-4e86-8e56-dde91889baba None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:11.734782 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f03919-c2db-4e86-8e56-dde91889baba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:11.741728 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f03919-c2db-4e86-8e56-dde91889baba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:11.743214 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f03919-c2db-4e86-8e56-dde91889baba None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:11.754836 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-a6f03919-c2db-4e86-8e56-dde91889baba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:09:11.754836 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:09:11.754836 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:09:11.754836 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:09:11.754836 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:09:11.754836 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:09:11.754836 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:09:11.754836 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:09:11.754836 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:09:11.754836 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:09:11.754836 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:09:11.754836 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:09:11.754836 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 05:09:11.754836 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 05:09:11.754836 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 05:09:11.755835 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 05:09:11.755835 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:09:11.755835 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:09:11.755835 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:09:11.755835 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:09:11.755835 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:09:11.755835 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:09:11.755835 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:09:11.755835 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:09:11.755835 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:09:11.755835 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:09:11.755835 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:09:11.755835 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:09:11.755835 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:09:11.755835 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:09:11.755835 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:09:11.755835 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:09:11.756731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 05:09:11.756731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 05:09:11.756731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 05:09:11.756731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 05:09:11.756731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 05:09:11.756731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 05:09:11.756731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:09:11.756731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:09:11.756731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 606/1230] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:09:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:09:11.762254 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910fac9-2209-439a-bfaa-288768cb1353 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:11.862619 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910fac9-2209-439a-bfaa-288768cb1353 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:11.897426 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7256138-2057-40e5-9d96-3c286fc4924c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:11.898087 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7256138-2057-40e5-9d96-3c286fc4924c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:11.898365 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7256138-2057-40e5-9d96-3c286fc4924c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:11.898634 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7256138-2057-40e5-9d96-3c286fc4924c None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:11.899324 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7256138-2057-40e5-9d96-3c286fc4924c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:09:11.899638 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7256138-2057-40e5-9d96-3c286fc4924c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:11.901037 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7256138-2057-40e5-9d96-3c286fc4924c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:11.913002 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 625/1231] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 05:09:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:09:11.924716 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f03919-c2db-4e86-8e56-dde91889baba None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:12.006777 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:12.051074 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f03919-c2db-4e86-8e56-dde91889baba None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:12.093814 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395b9b5-4fae-46c0-9943-3f4472131687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:12.094564 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395b9b5-4fae-46c0-9943-3f4472131687 None admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:12.094876 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395b9b5-4fae-46c0-9943-3f4472131687 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:12.095188 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395b9b5-4fae-46c0-9943-3f4472131687 None admin] PATH_INFO: `/v3/projects/e01ce1b1282e44a39d659b87f120995e/users/bd4387035099416789009f0ecb8ad282/roles/207bf98e31774cc0b21249de6d96155b` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:12.111314 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7395b9b5-4fae-46c0-9943-3f4472131687 None admin] RBAC: Authorizing `identity:create_grant(project_id=e01ce1b1282e44a39d659b87f120995e, user_id=bd4387035099416789009f0ecb8ad282, role_id=207bf98e31774cc0b21249de6d96155b)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:12.112889 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7395b9b5-4fae-46c0-9943-3f4472131687 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:12.128049 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 607/1232] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 05:09:11 2022] PUT /identity/v3/projects/e01ce1b1282e44a39d659b87f120995e/users/bd4387035099416789009f0ecb8ad282/roles/207bf98e31774cc0b21249de6d96155b => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:09:14.171586 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6465d85-da44-4637-a4a0-47e562db007d None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:14.172124 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6465d85-da44-4637-a4a0-47e562db007d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:14.172257 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6465d85-da44-4637-a4a0-47e562db007d None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:14.173516 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 626/1233] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:09:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:09:14.178691 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b90f3-ed35-41ef-a73e-c68b383a0e13 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:14.179168 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b90f3-ed35-41ef-a73e-c68b383a0e13 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:14.179322 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b90f3-ed35-41ef-a73e-c68b383a0e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:14.198835 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-841b90f3-ed35-41ef-a73e-c68b383a0e13 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:09:14.199997 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-841b90f3-ed35-41ef-a73e-c68b383a0e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:09:14.284584 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 608/1234] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:09:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:09:14.603956 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6465d85-da44-4637-a4a0-47e562db007d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:14.635611 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:14.731283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6465d85-da44-4637-a4a0-47e562db007d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:14.767127 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8991a57f-4fa9-439f-ae06-be7e6df1d2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:14.767750 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991a57f-4fa9-439f-ae06-be7e6df1d2bc None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:14.768021 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991a57f-4fa9-439f-ae06-be7e6df1d2bc None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:14.768285 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991a57f-4fa9-439f-ae06-be7e6df1d2bc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:14.778429 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8991a57f-4fa9-439f-ae06-be7e6df1d2bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:09:14.783372 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-8991a57f-4fa9-439f-ae06-be7e6df1d2bc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 05:09:14.783372 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:09:14.783372 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:09:14.783372 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:09:14.783372 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:09:14.783372 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:09:14.783372 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:09:14.783372 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:09:14.783372 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:09:14.783372 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:09:14.783372 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:09:14.783372 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:09:14.783372 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:09:14.783372 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:09:14.783372 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:09:14.783372 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:09:14.784541 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:09:14.784541 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:09:14.784541 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:09:14.784541 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:09:14.784541 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:09:14.784541 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:09:14.784541 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:09:14.784541 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:09:14.784541 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:09:14.784541 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:09:14.784541 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:09:14.784541 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:09:14.784541 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:09:14.784541 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:09:14.784541 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:09:14.784541 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:09:14.785546 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:09:14.785546 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:09:14.785546 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:09:14.785546 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:09:14.785546 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:09:14.785546 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:09:14.785546 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:09:14.785546 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:09:14.785546 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:09:14.785546 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:09:14.785546 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 05:09:14.785546 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:09:14.785546 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 627/1235] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Aug 31 05:09:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:09:14.790795 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b90f3-ed35-41ef-a73e-c68b383a0e13 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:14.837226 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:14.897361 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b90f3-ed35-41ef-a73e-c68b383a0e13 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:14.935433 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c8325-0013-4c06-867d-c176cafa5e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:14.935940 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c8325-0013-4c06-867d-c176cafa5e12 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:14.936212 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c8325-0013-4c06-867d-c176cafa5e12 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:14.936490 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c8325-0013-4c06-867d-c176cafa5e12 None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:14.937164 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533c8325-0013-4c06-867d-c176cafa5e12 None admin] RBAC: Adding query filter params (name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:09:14.937508 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533c8325-0013-4c06-867d-c176cafa5e12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:14.941074 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533c8325-0013-4c06-867d-c176cafa5e12 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:14.950600 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 609/1236] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Aug 31 05:09:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:09:14.956575 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8991a57f-4fa9-439f-ae06-be7e6df1d2bc None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:15.056748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:15.073640 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8991a57f-4fa9-439f-ae06-be7e6df1d2bc None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:15.108656 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3729d080-6626-42a9-be4a-23be1f060431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:15.109297 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729d080-6626-42a9-be4a-23be1f060431 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:15.109573 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729d080-6626-42a9-be4a-23be1f060431 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:15.109838 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729d080-6626-42a9-be4a-23be1f060431 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:15.120835 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3729d080-6626-42a9-be4a-23be1f060431 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:15.122401 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3729d080-6626-42a9-be4a-23be1f060431 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:15.137774 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-3729d080-6626-42a9-be4a-23be1f060431 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:09:15.137774 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:09:15.137774 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:09:15.137774 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:09:15.137774 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:09:15.137774 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:09:15.137774 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:09:15.137774 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:09:15.137774 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:09:15.137774 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:09:15.137774 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:09:15.137774 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:09:15.137774 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:09:15.137774 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:09:15.137774 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:09:15.137774 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:09:15.138791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:09:15.138791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:09:15.138791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:09:15.138791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:09:15.138791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:09:15.138791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:09:15.138791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:09:15.138791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:09:15.138791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:09:15.138791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:09:15.138791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:09:15.138791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:09:15.138791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:09:15.138791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:09:15.138791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:09:15.138791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:09:15.138791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:09:15.139710 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:09:15.139710 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:09:15.139710 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:09:15.139710 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 628/1237] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:09:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:09:15.145844 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c8325-0013-4c06-867d-c176cafa5e12 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:15.253770 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c8325-0013-4c06-867d-c176cafa5e12 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:15.264769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:15.292399 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd22d58-73c5-46d2-958c-7bdaf82fbf6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:15.292998 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd22d58-73c5-46d2-958c-7bdaf82fbf6a None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:15.293267 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd22d58-73c5-46d2-958c-7bdaf82fbf6a None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:15.293529 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd22d58-73c5-46d2-958c-7bdaf82fbf6a None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:15.294194 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd22d58-73c5-46d2-958c-7bdaf82fbf6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:09:15.294487 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd22d58-73c5-46d2-958c-7bdaf82fbf6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:15.295853 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd22d58-73c5-46d2-958c-7bdaf82fbf6a None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:15.308009 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 610/1238] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:09:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:09:15.313844 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3729d080-6626-42a9-be4a-23be1f060431 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:15.425805 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3729d080-6626-42a9-be4a-23be1f060431 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:15.460425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358cf1e-cff4-4431-ae57-cfcd81dc12a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:15.461143 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358cf1e-cff4-4431-ae57-cfcd81dc12a7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:15.461425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358cf1e-cff4-4431-ae57-cfcd81dc12a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:15.461680 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358cf1e-cff4-4431-ae57-cfcd81dc12a7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:15.464361 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:15.469870 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358cf1e-cff4-4431-ae57-cfcd81dc12a7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:15.471339 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358cf1e-cff4-4431-ae57-cfcd81dc12a7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:15.482522 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-1358cf1e-cff4-4431-ae57-cfcd81dc12a7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 05:09:15.482522 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:09:15.482522 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:09:15.482522 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:09:15.482522 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:09:15.482522 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:09:15.482522 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:09:15.482522 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:09:15.482522 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:09:15.482522 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:09:15.482522 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:09:15.482522 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:09:15.482522 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 05:09:15.482522 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 05:09:15.482522 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 05:09:15.482522 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 05:09:15.483689 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:09:15.483689 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:09:15.483689 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 05:09:15.483689 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:09:15.483689 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 05:09:15.483689 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:09:15.483689 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:09:15.483689 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:09:15.483689 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:09:15.483689 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:09:15.483689 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:09:15.483689 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:09:15.483689 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:09:15.483689 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:09:15.483689 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:09:15.483689 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:09:15.483689 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:09:15.483689 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:09:15.484861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:09:15.484861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:09:15.484861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 05:09:15.484861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 05:09:15.484861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 05:09:15.484861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 05:09:15.484861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 05:09:15.484861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 05:09:15.484861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 05:09:15.484861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:09:15.484861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 629/1239] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:09:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 05:09:15.489946 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd22d58-73c5-46d2-958c-7bdaf82fbf6a None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:15.605255 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd22d58-73c5-46d2-958c-7bdaf82fbf6a None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:15.639234 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472d1269-3258-4861-9067-75fe00c0ab71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:15.639790 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d1269-3258-4861-9067-75fe00c0ab71 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:15.640063 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d1269-3258-4861-9067-75fe00c0ab71 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:15.640325 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d1269-3258-4861-9067-75fe00c0ab71 None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:15.641069 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472d1269-3258-4861-9067-75fe00c0ab71 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:09:15.641379 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472d1269-3258-4861-9067-75fe00c0ab71 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:15.642847 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472d1269-3258-4861-9067-75fe00c0ab71 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:15.662420 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 611/1240] 10.222.0.36 () {56 vars in 1143 bytes} [Wed Aug 31 05:09:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:09:15.668504 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358cf1e-cff4-4431-ae57-cfcd81dc12a7 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:15.778857 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358cf1e-cff4-4431-ae57-cfcd81dc12a7 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:15.818029 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb9d234-c26f-48ac-b57c-66ef5dcd2a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:15.818675 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9d234-c26f-48ac-b57c-66ef5dcd2a98 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:15.818956 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9d234-c26f-48ac-b57c-66ef5dcd2a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:15.819259 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9d234-c26f-48ac-b57c-66ef5dcd2a98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:15.893606 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:15.900874 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb9d234-c26f-48ac-b57c-66ef5dcd2a98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:15.902434 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb9d234-c26f-48ac-b57c-66ef5dcd2a98 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:16.024740 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 630/1241] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:09:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 05:09:16.030981 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472d1269-3258-4861-9067-75fe00c0ab71 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:16.061469 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:16.155478 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472d1269-3258-4861-9067-75fe00c0ab71 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:16.198787 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5394db-f7d8-4f2a-ab9b-4edc80938179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:16.199465 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5394db-f7d8-4f2a-ab9b-4edc80938179 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:16.199803 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5394db-f7d8-4f2a-ab9b-4edc80938179 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:16.200191 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5394db-f7d8-4f2a-ab9b-4edc80938179 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:16.213116 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5394db-f7d8-4f2a-ab9b-4edc80938179 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:16.214628 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5394db-f7d8-4f2a-ab9b-4edc80938179 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:16.230699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-8b5394db-f7d8-4f2a-ab9b-4edc80938179 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:09:16.230699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:09:16.230699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:09:16.230699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:09:16.230699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:09:16.230699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:09:16.230699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:09:16.230699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:09:16.230699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:09:16.230699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:09:16.230699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:09:16.230699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:09:16.230699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:09:16.230699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:09:16.230699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:09:16.230699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:09:16.231955 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:09:16.231955 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:09:16.231955 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:09:16.231955 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:09:16.231955 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:09:16.231955 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:09:16.231955 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:09:16.231955 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:09:16.231955 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:09:16.231955 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:09:16.231955 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:09:16.231955 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:09:16.231955 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:09:16.231955 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:09:16.231955 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:09:16.231955 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:09:16.231955 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:09:16.232972 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:09:16.232972 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:09:16.232972 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:09:16.232972 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 612/1242] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:09:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:09:16.238167 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb9d234-c26f-48ac-b57c-66ef5dcd2a98 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:16.337889 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:16.344998 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb9d234-c26f-48ac-b57c-66ef5dcd2a98 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:16.385289 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ae3f79-208c-496f-bdd2-c92d0aac8e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:16.386205 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae3f79-208c-496f-bdd2-c92d0aac8e00 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:16.386539 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae3f79-208c-496f-bdd2-c92d0aac8e00 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:16.386868 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae3f79-208c-496f-bdd2-c92d0aac8e00 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:16.387797 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ae3f79-208c-496f-bdd2-c92d0aac8e00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:09:16.388147 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ae3f79-208c-496f-bdd2-c92d0aac8e00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:16.389797 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ae3f79-208c-496f-bdd2-c92d0aac8e00 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:16.403333 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 631/1243] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:09:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:09:16.410233 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5394db-f7d8-4f2a-ab9b-4edc80938179 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:16.495147 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:16.519936 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5394db-f7d8-4f2a-ab9b-4edc80938179 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:16.555952 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea72da-0742-427a-b4b3-2947f78b016a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:16.556577 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea72da-0742-427a-b4b3-2947f78b016a None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:16.556850 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea72da-0742-427a-b4b3-2947f78b016a None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:16.557119 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea72da-0742-427a-b4b3-2947f78b016a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:16.564943 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfea72da-0742-427a-b4b3-2947f78b016a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:16.566455 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfea72da-0742-427a-b4b3-2947f78b016a None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:16.579008 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-bfea72da-0742-427a-b4b3-2947f78b016a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:09:16.579008 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:09:16.579008 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:09:16.579008 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:09:16.579008 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:09:16.579008 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:09:16.579008 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:09:16.579008 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:09:16.579008 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:09:16.579008 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:09:16.579008 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:09:16.579008 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:09:16.579008 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 05:09:16.579008 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 05:09:16.579008 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 05:09:16.580146 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 05:09:16.580146 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:09:16.580146 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:09:16.580146 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:09:16.580146 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:09:16.580146 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:09:16.580146 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:09:16.580146 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:09:16.580146 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:09:16.580146 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:09:16.580146 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:09:16.580146 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:09:16.580146 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:09:16.580146 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:09:16.580146 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:09:16.580146 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:09:16.580146 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:09:16.581423 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 05:09:16.581423 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 05:09:16.581423 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 05:09:16.581423 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 05:09:16.581423 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 05:09:16.581423 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 05:09:16.581423 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:09:16.581423 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:09:16.581423 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 613/1244] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:09:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:09:16.586739 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ae3f79-208c-496f-bdd2-c92d0aac8e00 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:16.696511 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ae3f79-208c-496f-bdd2-c92d0aac8e00 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:16.732329 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c507e3cc-dcf9-41f8-bf39-e528b75533ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:16.732936 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507e3cc-dcf9-41f8-bf39-e528b75533ce None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:16.733202 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507e3cc-dcf9-41f8-bf39-e528b75533ce None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:16.733464 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507e3cc-dcf9-41f8-bf39-e528b75533ce None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:16.734178 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c507e3cc-dcf9-41f8-bf39-e528b75533ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:09:16.734472 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c507e3cc-dcf9-41f8-bf39-e528b75533ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:16.735892 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c507e3cc-dcf9-41f8-bf39-e528b75533ce None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:16.750212 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 632/1245] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 05:09:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:09:16.757248 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea72da-0742-427a-b4b3-2947f78b016a None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:16.872293 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea72da-0742-427a-b4b3-2947f78b016a None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:16.915023 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2fb3b5-9d4d-46a9-85c9-c9d20afa868d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:16.915622 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2fb3b5-9d4d-46a9-85c9-c9d20afa868d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:16.915895 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2fb3b5-9d4d-46a9-85c9-c9d20afa868d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:16.916145 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2fb3b5-9d4d-46a9-85c9-c9d20afa868d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:16.916958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2fb3b5-9d4d-46a9-85c9-c9d20afa868d None admin] RBAC: Adding query filter params (role.id=207bf98e31774cc0b21249de6d96155b, scope.project.id=e01ce1b1282e44a39d659b87f120995e, user.id=bd4387035099416789009f0ecb8ad282) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:09:16.917290 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2fb3b5-9d4d-46a9-85c9-c9d20afa868d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:16.918731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2fb3b5-9d4d-46a9-85c9-c9d20afa868d None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:16.924783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 614/1246] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:09:16 2022] GET /identity/v3/role_assignments?user.id=bd4387035099416789009f0ecb8ad282&scope.project.id=e01ce1b1282e44a39d659b87f120995e&role.id=207bf98e31774cc0b21249de6d96155b => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:09:18.874258 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21f1e6-4e72-44f5-9581-2d424e221e86 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:18.874990 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21f1e6-4e72-44f5-9581-2d424e221e86 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:18.874990 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21f1e6-4e72-44f5-9581-2d424e221e86 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:18.876331 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 633/1247] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:09:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:09:18.882510 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afdd09b-5272-481d-8b57-480be8b6be30 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:18.882989 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afdd09b-5272-481d-8b57-480be8b6be30 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:18.883094 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afdd09b-5272-481d-8b57-480be8b6be30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:18.889357 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:18.903429 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-3afdd09b-5272-481d-8b57-480be8b6be30 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:09:18.904860 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3afdd09b-5272-481d-8b57-480be8b6be30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:09:18.993162 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 615/1248] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:09:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:09:19.259481 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21f1e6-4e72-44f5-9581-2d424e221e86 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:19.280923 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:19.379420 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21f1e6-4e72-44f5-9581-2d424e221e86 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:19.417524 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a46bcbd-668c-40a9-82a0-f7f2713ba884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:19.418214 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46bcbd-668c-40a9-82a0-f7f2713ba884 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:19.418480 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46bcbd-668c-40a9-82a0-f7f2713ba884 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:19.418766 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46bcbd-668c-40a9-82a0-f7f2713ba884 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:19.431341 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a46bcbd-668c-40a9-82a0-f7f2713ba884 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:09:19.437032 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-1a46bcbd-668c-40a9-82a0-f7f2713ba884 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:09:19.437032 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:09:19.437032 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:09:19.437032 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:09:19.437032 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:09:19.437032 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:09:19.437032 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:09:19.437032 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:09:19.437032 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:09:19.437032 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:09:19.437032 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:09:19.437032 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:09:19.437032 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:09:19.437032 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:09:19.437032 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:09:19.437032 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:09:19.438209 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:09:19.438209 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:09:19.438209 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:09:19.438209 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:09:19.438209 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:09:19.438209 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:09:19.438209 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:09:19.438209 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:09:19.438209 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:09:19.438209 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:09:19.438209 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:09:19.438209 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:09:19.438209 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:09:19.438209 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:09:19.438209 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:09:19.438209 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:09:19.439271 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:09:19.439271 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:09:19.439271 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:09:19.439271 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:09:19.439271 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:09:19.439271 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:09:19.439271 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:09:19.439271 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:09:19.439271 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:09:19.439271 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:09:19.439271 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:09:19.439271 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:09:19.439271 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 634/1249] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 05:09:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:09:19.444764 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afdd09b-5272-481d-8b57-480be8b6be30 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:19.559823 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afdd09b-5272-481d-8b57-480be8b6be30 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:19.588781 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:19.595309 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b971b-a40d-470f-91b9-dd39f2eb387c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:19.595884 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b971b-a40d-470f-91b9-dd39f2eb387c None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:19.596159 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b971b-a40d-470f-91b9-dd39f2eb387c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:19.596432 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b971b-a40d-470f-91b9-dd39f2eb387c None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:19.597083 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785b971b-a40d-470f-91b9-dd39f2eb387c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:09:19.597378 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785b971b-a40d-470f-91b9-dd39f2eb387c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:19.598816 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785b971b-a40d-470f-91b9-dd39f2eb387c None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:19.607464 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 616/1250] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:09:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:09:19.613878 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a46bcbd-668c-40a9-82a0-f7f2713ba884 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:19.708345 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:19.736431 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a46bcbd-668c-40a9-82a0-f7f2713ba884 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:19.778017 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3952da78-25ba-4ee6-9512-19216764f541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:19.778853 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952da78-25ba-4ee6-9512-19216764f541 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:19.779126 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952da78-25ba-4ee6-9512-19216764f541 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:19.779393 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952da78-25ba-4ee6-9512-19216764f541 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:19.791451 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3952da78-25ba-4ee6-9512-19216764f541 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:19.792981 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3952da78-25ba-4ee6-9512-19216764f541 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:19.810458 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-3952da78-25ba-4ee6-9512-19216764f541 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:09:19.810458 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:09:19.810458 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:09:19.810458 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:09:19.810458 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:09:19.810458 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:09:19.810458 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:09:19.810458 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:09:19.810458 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:09:19.810458 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:09:19.810458 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:09:19.810458 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:09:19.810458 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:09:19.810458 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:09:19.810458 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:09:19.810458 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:09:19.811890 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:09:19.811890 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:09:19.811890 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:09:19.811890 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:09:19.811890 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:09:19.811890 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:09:19.811890 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:09:19.811890 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:09:19.811890 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:09:19.811890 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:09:19.811890 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:09:19.811890 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:09:19.811890 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:09:19.811890 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:09:19.811890 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:09:19.811890 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:09:19.811890 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:09:19.812989 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:09:19.812989 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:09:19.812989 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:09:19.812989 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 635/1251] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:09:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:09:19.818152 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b971b-a40d-470f-91b9-dd39f2eb387c None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:19.939879 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b971b-a40d-470f-91b9-dd39f2eb387c None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:19.985753 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f4976-6f28-4705-897e-f2255b35d5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:19.986467 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f4976-6f28-4705-897e-f2255b35d5b6 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:19.986834 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f4976-6f28-4705-897e-f2255b35d5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:19.987180 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f4976-6f28-4705-897e-f2255b35d5b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:19.988038 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f4976-6f28-4705-897e-f2255b35d5b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:09:19.988808 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f4976-6f28-4705-897e-f2255b35d5b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:19.990697 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f4976-6f28-4705-897e-f2255b35d5b6 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:20.005605 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 617/1252] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:09:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:09:20.011849 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3952da78-25ba-4ee6-9512-19216764f541 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:20.113048 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3952da78-25ba-4ee6-9512-19216764f541 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:20.139287 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:20.147774 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05c83d-a916-4255-bd69-9176237b3eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:20.148458 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05c83d-a916-4255-bd69-9176237b3eee None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:20.148727 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05c83d-a916-4255-bd69-9176237b3eee None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:20.148993 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05c83d-a916-4255-bd69-9176237b3eee None admin] PATH_INFO: `/v3/users/placement` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:20.157297 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d05c83d-a916-4255-bd69-9176237b3eee None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:20.158836 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d05c83d-a916-4255-bd69-9176237b3eee None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:20.170469 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-4d05c83d-a916-4255-bd69-9176237b3eee None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 05:09:20.170469 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:09:20.170469 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:09:20.170469 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:09:20.170469 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:09:20.170469 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:09:20.170469 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:09:20.170469 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:09:20.170469 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:09:20.170469 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:09:20.170469 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:09:20.170469 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:09:20.170469 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 05:09:20.170469 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 05:09:20.170469 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 05:09:20.170469 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 05:09:20.171758 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:09:20.171758 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:09:20.171758 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 05:09:20.171758 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:09:20.171758 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 05:09:20.171758 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:09:20.171758 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:09:20.171758 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:09:20.171758 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:09:20.171758 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:09:20.171758 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:09:20.171758 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:09:20.171758 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:09:20.171758 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:09:20.171758 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:09:20.171758 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:09:20.171758 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:09:20.171758 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:09:20.172992 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:09:20.172992 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:09:20.172992 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 05:09:20.172992 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 05:09:20.172992 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 05:09:20.172992 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 05:09:20.172992 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 05:09:20.172992 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 05:09:20.172992 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 05:09:20.172992 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:09:20.172992 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 636/1253] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:09:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:09:20.177790 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f4976-6f28-4705-897e-f2255b35d5b6 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:20.236096 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:20.286964 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f4976-6f28-4705-897e-f2255b35d5b6 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:20.319931 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74db646b-57f4-4eea-ad4b-d4d1e93b1000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:20.320913 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db646b-57f4-4eea-ad4b-d4d1e93b1000 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:20.321223 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db646b-57f4-4eea-ad4b-d4d1e93b1000 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:20.321528 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db646b-57f4-4eea-ad4b-d4d1e93b1000 None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:20.322426 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74db646b-57f4-4eea-ad4b-d4d1e93b1000 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:09:20.322777 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74db646b-57f4-4eea-ad4b-d4d1e93b1000 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:20.324492 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74db646b-57f4-4eea-ad4b-d4d1e93b1000 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:20.350552 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 618/1254] 10.222.0.36 () {56 vars in 1143 bytes} [Wed Aug 31 05:09:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:09:20.351952 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05c83d-a916-4255-bd69-9176237b3eee None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:20.468126 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05c83d-a916-4255-bd69-9176237b3eee None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:20.509755 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc920b2-8dbf-4e8c-90fa-733c8be5fb9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:20.510408 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc920b2-8dbf-4e8c-90fa-733c8be5fb9e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:20.510710 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc920b2-8dbf-4e8c-90fa-733c8be5fb9e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:20.510974 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc920b2-8dbf-4e8c-90fa-733c8be5fb9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:20.571884 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:20.587584 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc920b2-8dbf-4e8c-90fa-733c8be5fb9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:20.589036 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc920b2-8dbf-4e8c-90fa-733c8be5fb9e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:20.702041 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 637/1255] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:09:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 05:09:20.707989 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74db646b-57f4-4eea-ad4b-d4d1e93b1000 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:20.815906 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74db646b-57f4-4eea-ad4b-d4d1e93b1000 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:20.829439 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:20.854646 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2446b5e-f50f-4387-97c3-6c41da25c124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:20.855281 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2446b5e-f50f-4387-97c3-6c41da25c124 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:20.855665 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2446b5e-f50f-4387-97c3-6c41da25c124 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:20.856024 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2446b5e-f50f-4387-97c3-6c41da25c124 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:20.868091 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2446b5e-f50f-4387-97c3-6c41da25c124 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:20.869958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2446b5e-f50f-4387-97c3-6c41da25c124 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:20.886945 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-d2446b5e-f50f-4387-97c3-6c41da25c124 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:09:20.886945 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:09:20.886945 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:09:20.886945 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:09:20.886945 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:09:20.886945 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:09:20.886945 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:09:20.886945 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:09:20.886945 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:09:20.886945 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:09:20.886945 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:09:20.886945 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:09:20.886945 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:09:20.886945 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:09:20.886945 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:09:20.886945 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:09:20.888295 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:09:20.888295 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:09:20.888295 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:09:20.888295 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:09:20.888295 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:09:20.888295 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:09:20.888295 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:09:20.888295 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:09:20.888295 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:09:20.888295 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:09:20.888295 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:09:20.888295 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:09:20.888295 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:09:20.888295 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:09:20.888295 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:09:20.888295 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:09:20.888295 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:09:20.889559 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:09:20.889559 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:09:20.889559 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:09:20.889559 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 619/1256] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:09:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:09:20.895488 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc920b2-8dbf-4e8c-90fa-733c8be5fb9e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:21.010697 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:21.022945 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc920b2-8dbf-4e8c-90fa-733c8be5fb9e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:21.064790 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb7781-f1fb-4f44-bcef-8bf0361714e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:21.065530 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb7781-f1fb-4f44-bcef-8bf0361714e5 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:21.065873 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb7781-f1fb-4f44-bcef-8bf0361714e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:21.066298 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb7781-f1fb-4f44-bcef-8bf0361714e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:21.066914 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fb7781-f1fb-4f44-bcef-8bf0361714e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:09:21.067189 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fb7781-f1fb-4f44-bcef-8bf0361714e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:21.068565 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fb7781-f1fb-4f44-bcef-8bf0361714e5 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:21.080453 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 638/1257] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:09:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:09:21.087469 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2446b5e-f50f-4387-97c3-6c41da25c124 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:21.199866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2446b5e-f50f-4387-97c3-6c41da25c124 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:21.235889 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05d3ac-4a25-4b1b-8d30-2c62470cccc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:21.236592 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05d3ac-4a25-4b1b-8d30-2c62470cccc9 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:21.236865 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05d3ac-4a25-4b1b-8d30-2c62470cccc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:21.237129 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05d3ac-4a25-4b1b-8d30-2c62470cccc9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:21.244754 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c05d3ac-4a25-4b1b-8d30-2c62470cccc9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:21.246464 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c05d3ac-4a25-4b1b-8d30-2c62470cccc9 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:21.259772 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-7c05d3ac-4a25-4b1b-8d30-2c62470cccc9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:09:21.259772 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:09:21.259772 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:09:21.259772 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:09:21.259772 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:09:21.259772 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:09:21.259772 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:09:21.259772 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:09:21.259772 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:09:21.259772 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:09:21.259772 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:09:21.259772 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:09:21.259772 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 05:09:21.259772 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 05:09:21.259772 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 05:09:21.261039 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 05:09:21.261039 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:09:21.261039 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:09:21.261039 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:09:21.261039 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:09:21.261039 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:09:21.261039 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:09:21.261039 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:09:21.261039 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:09:21.261039 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:09:21.261039 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:09:21.261039 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:09:21.261039 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:09:21.261039 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:09:21.261039 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:09:21.261039 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:09:21.261039 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:09:21.262533 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 05:09:21.262533 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 05:09:21.262533 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 05:09:21.262533 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 05:09:21.262533 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 05:09:21.262533 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 05:09:21.262533 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:09:21.262533 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:09:21.262533 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 620/1258] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:09:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:09:21.267947 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb7781-f1fb-4f44-bcef-8bf0361714e5 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:21.373612 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb7781-f1fb-4f44-bcef-8bf0361714e5 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:21.407773 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf8ed0b-b009-4a5e-91b9-c7358b2913a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:21.408373 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf8ed0b-b009-4a5e-91b9-c7358b2913a9 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:21.408765 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf8ed0b-b009-4a5e-91b9-c7358b2913a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:21.408883 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf8ed0b-b009-4a5e-91b9-c7358b2913a9 None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:21.409573 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf8ed0b-b009-4a5e-91b9-c7358b2913a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:09:21.409889 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf8ed0b-b009-4a5e-91b9-c7358b2913a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:21.411334 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf8ed0b-b009-4a5e-91b9-c7358b2913a9 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:21.423379 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 639/1259] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 05:09:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:09:21.429990 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05d3ac-4a25-4b1b-8d30-2c62470cccc9 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:21.470107 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:21.564210 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05d3ac-4a25-4b1b-8d30-2c62470cccc9 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:21.602190 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e656ad8-e04b-4a05-823a-3dbea0f30baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:21.602800 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e656ad8-e04b-4a05-823a-3dbea0f30baf None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:21.603075 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e656ad8-e04b-4a05-823a-3dbea0f30baf None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:21.603343 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e656ad8-e04b-4a05-823a-3dbea0f30baf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:21.604073 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e656ad8-e04b-4a05-823a-3dbea0f30baf None admin] RBAC: Adding query filter params (role.id=c3d5589dd2864be9a0f26f5802d874cd, scope.project.id=e01ce1b1282e44a39d659b87f120995e, user.id=bd4387035099416789009f0ecb8ad282) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:09:21.604379 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e656ad8-e04b-4a05-823a-3dbea0f30baf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:21.605976 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e656ad8-e04b-4a05-823a-3dbea0f30baf None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:21.614081 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 621/1260] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:09:21 2022] GET /identity/v3/role_assignments?user.id=bd4387035099416789009f0ecb8ad282&scope.project.id=e01ce1b1282e44a39d659b87f120995e&role.id=c3d5589dd2864be9a0f26f5802d874cd => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:09:23.507577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde6b10-3723-46e9-87d1-2cf8dc6c57d4 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:23.508134 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde6b10-3723-46e9-87d1-2cf8dc6c57d4 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:23.508134 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde6b10-3723-46e9-87d1-2cf8dc6c57d4 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:23.509463 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 640/1261] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:09:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:09:23.515768 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530b52a-6734-4c69-9d73-931b7fd806ee None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:23.516543 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530b52a-6734-4c69-9d73-931b7fd806ee None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:23.516710 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530b52a-6734-4c69-9d73-931b7fd806ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:23.539665 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-3530b52a-6734-4c69-9d73-931b7fd806ee None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:09:23.540839 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3530b52a-6734-4c69-9d73-931b7fd806ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:09:23.612581 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 622/1262] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:09:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:09:23.884876 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dde6b10-3723-46e9-87d1-2cf8dc6c57d4 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:23.897485 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:24.002124 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dde6b10-3723-46e9-87d1-2cf8dc6c57d4 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:24.039238 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5707a8e-ff7f-4114-a65a-3a066a9adf18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:24.039840 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5707a8e-ff7f-4114-a65a-3a066a9adf18 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:24.040109 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5707a8e-ff7f-4114-a65a-3a066a9adf18 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:24.040379 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5707a8e-ff7f-4114-a65a-3a066a9adf18 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:24.050608 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5707a8e-ff7f-4114-a65a-3a066a9adf18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:09:24.055494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-d5707a8e-ff7f-4114-a65a-3a066a9adf18 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:09:24.055494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:09:24.055494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:09:24.055494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:09:24.055494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:09:24.055494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:09:24.055494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:09:24.055494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:09:24.055494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:09:24.055494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:09:24.055494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:09:24.055494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:09:24.055494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:09:24.055494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:09:24.055494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:09:24.055494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:09:24.056570 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:09:24.056570 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:09:24.056570 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:09:24.056570 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:09:24.056570 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:09:24.056570 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:09:24.056570 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:09:24.056570 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:09:24.056570 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:09:24.056570 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:09:24.056570 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:09:24.056570 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:09:24.056570 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:09:24.056570 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:09:24.056570 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:09:24.056570 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:09:24.057544 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:09:24.057544 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:09:24.057544 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:09:24.057544 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:09:24.057544 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:09:24.057544 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:09:24.057544 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:09:24.057544 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:09:24.057544 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:09:24.057544 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:09:24.057544 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:09:24.057544 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:09:24.057544 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 641/1263] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 05:09:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:09:24.063401 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3530b52a-6734-4c69-9d73-931b7fd806ee None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:24.090815 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:24.183813 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3530b52a-6734-4c69-9d73-931b7fd806ee None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:24.222673 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a172a781-68aa-44a0-90f2-91af2cedcefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:24.223599 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172a781-68aa-44a0-90f2-91af2cedcefd None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:24.224030 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172a781-68aa-44a0-90f2-91af2cedcefd None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:24.224436 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172a781-68aa-44a0-90f2-91af2cedcefd None admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:24.225478 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a172a781-68aa-44a0-90f2-91af2cedcefd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:09:24.225940 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a172a781-68aa-44a0-90f2-91af2cedcefd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:24.228107 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a172a781-68aa-44a0-90f2-91af2cedcefd None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:24.241152 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 623/1264] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:09:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:09:24.247068 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5707a8e-ff7f-4114-a65a-3a066a9adf18 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:24.316445 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:24.359105 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5707a8e-ff7f-4114-a65a-3a066a9adf18 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:24.396055 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758a032-e5d7-4a31-94b0-b9ce4e15f346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:24.396981 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758a032-e5d7-4a31-94b0-b9ce4e15f346 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:24.397427 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758a032-e5d7-4a31-94b0-b9ce4e15f346 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:24.397694 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758a032-e5d7-4a31-94b0-b9ce4e15f346 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:24.410121 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7758a032-e5d7-4a31-94b0-b9ce4e15f346 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:24.411567 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7758a032-e5d7-4a31-94b0-b9ce4e15f346 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:24.427155 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-7758a032-e5d7-4a31-94b0-b9ce4e15f346 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:09:24.427155 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:09:24.427155 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:09:24.427155 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:09:24.427155 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:09:24.427155 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:09:24.427155 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:09:24.427155 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:09:24.427155 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:09:24.427155 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:09:24.427155 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:09:24.427155 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:09:24.427155 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:09:24.427155 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:09:24.427155 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:09:24.427155 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:09:24.428194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:09:24.428194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:09:24.428194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:09:24.428194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:09:24.428194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:09:24.428194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:09:24.428194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:09:24.428194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:09:24.428194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:09:24.428194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:09:24.428194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:09:24.428194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:09:24.428194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:09:24.428194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:09:24.428194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:09:24.428194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:09:24.428194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:09:24.429240 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:09:24.429240 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:09:24.429240 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:09:24.429240 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 642/1265] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:09:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:09:24.434292 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a172a781-68aa-44a0-90f2-91af2cedcefd None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:24.523845 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:24.544108 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a172a781-68aa-44a0-90f2-91af2cedcefd None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:24.578765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a5e1c3-f13d-44be-a30e-e6cfa9f13716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:24.579353 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5e1c3-f13d-44be-a30e-e6cfa9f13716 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:24.579627 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5e1c3-f13d-44be-a30e-e6cfa9f13716 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:24.579892 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5e1c3-f13d-44be-a30e-e6cfa9f13716 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:24.580525 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a5e1c3-f13d-44be-a30e-e6cfa9f13716 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:09:24.580819 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a5e1c3-f13d-44be-a30e-e6cfa9f13716 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:24.582214 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a5e1c3-f13d-44be-a30e-e6cfa9f13716 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:24.593870 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 624/1266] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:09:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:09:24.599789 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758a032-e5d7-4a31-94b0-b9ce4e15f346 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:24.722420 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758a032-e5d7-4a31-94b0-b9ce4e15f346 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:24.742356 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:24.758250 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5330dc7d-144e-454d-b26e-2ea41fe36a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:24.758890 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5330dc7d-144e-454d-b26e-2ea41fe36a0c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:24.759160 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5330dc7d-144e-454d-b26e-2ea41fe36a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:24.759420 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5330dc7d-144e-454d-b26e-2ea41fe36a0c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:24.766858 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5330dc7d-144e-454d-b26e-2ea41fe36a0c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:24.768284 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5330dc7d-144e-454d-b26e-2ea41fe36a0c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:24.779377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-5330dc7d-144e-454d-b26e-2ea41fe36a0c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 05:09:24.779377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:09:24.779377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:09:24.779377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:09:24.779377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:09:24.779377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:09:24.779377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:09:24.779377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:09:24.779377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:09:24.779377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:09:24.779377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:09:24.779377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:09:24.779377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 05:09:24.779377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 05:09:24.779377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 05:09:24.779377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 05:09:24.783800 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:09:24.783800 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:09:24.783800 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 05:09:24.783800 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:09:24.783800 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 05:09:24.783800 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:09:24.783800 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:09:24.783800 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:09:24.783800 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:09:24.783800 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:09:24.783800 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:09:24.783800 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:09:24.783800 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:09:24.783800 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:09:24.783800 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:09:24.783800 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:09:24.783800 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:09:24.783800 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:09:24.784938 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:09:24.784938 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:09:24.784938 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 05:09:24.784938 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 05:09:24.784938 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 05:09:24.784938 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 05:09:24.784938 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 05:09:24.784938 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 05:09:24.784938 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 05:09:24.784938 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:09:24.784938 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 643/1267] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:09:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:09:24.788286 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a5e1c3-f13d-44be-a30e-e6cfa9f13716 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:24.891919 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a5e1c3-f13d-44be-a30e-e6cfa9f13716 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:24.926937 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f5e5e2-0c19-4be3-b958-7bdd28df1354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:24.927505 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5e5e2-0c19-4be3-b958-7bdd28df1354 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:24.927780 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5e5e2-0c19-4be3-b958-7bdd28df1354 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:24.928041 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5e5e2-0c19-4be3-b958-7bdd28df1354 None admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:24.928793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f5e5e2-0c19-4be3-b958-7bdd28df1354 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:09:24.929085 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f5e5e2-0c19-4be3-b958-7bdd28df1354 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:24.930547 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f5e5e2-0c19-4be3-b958-7bdd28df1354 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:24.940071 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:24.949139 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 625/1268] 10.222.0.36 () {56 vars in 1143 bytes} [Wed Aug 31 05:09:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:09:24.955643 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5330dc7d-144e-454d-b26e-2ea41fe36a0c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:25.078451 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5330dc7d-144e-454d-b26e-2ea41fe36a0c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:25.112461 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8040aa1f-b978-4ce3-a00a-ee8936bf1a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:25.113386 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8040aa1f-b978-4ce3-a00a-ee8936bf1a63 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:25.113664 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8040aa1f-b978-4ce3-a00a-ee8936bf1a63 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:25.113926 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8040aa1f-b978-4ce3-a00a-ee8936bf1a63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:25.126394 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8040aa1f-b978-4ce3-a00a-ee8936bf1a63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:25.127820 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8040aa1f-b978-4ce3-a00a-ee8936bf1a63 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:25.141699 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-8040aa1f-b978-4ce3-a00a-ee8936bf1a63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:09:25.141699 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:09:25.141699 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:09:25.141699 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:09:25.141699 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:09:25.141699 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:09:25.141699 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:09:25.141699 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:09:25.141699 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:09:25.141699 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:09:25.141699 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:09:25.141699 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:09:25.141699 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:09:25.141699 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:09:25.141699 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:09:25.141699 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:09:25.142822 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:09:25.142822 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:09:25.142822 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:09:25.142822 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:09:25.142822 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:09:25.142822 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:09:25.142822 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:09:25.142822 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:09:25.142822 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:09:25.142822 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:09:25.142822 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:09:25.142822 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:09:25.142822 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:09:25.142822 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:09:25.142822 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:09:25.142822 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:09:25.142822 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:09:25.143735 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:09:25.143735 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:09:25.143735 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:09:25.143735 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 644/1269] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:09:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:09:25.150247 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f5e5e2-0c19-4be3-b958-7bdd28df1354 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:25.260224 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f5e5e2-0c19-4be3-b958-7bdd28df1354 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:25.294729 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4e9ff-ec3e-40bc-bd2f-ad256bb9ea4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:25.295316 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4e9ff-ec3e-40bc-bd2f-ad256bb9ea4d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:25.295592 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4e9ff-ec3e-40bc-bd2f-ad256bb9ea4d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:25.295854 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4e9ff-ec3e-40bc-bd2f-ad256bb9ea4d None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:25.296484 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f4e9ff-ec3e-40bc-bd2f-ad256bb9ea4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:09:25.296774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f4e9ff-ec3e-40bc-bd2f-ad256bb9ea4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:25.298163 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f4e9ff-ec3e-40bc-bd2f-ad256bb9ea4d None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:25.309298 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 626/1270] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:09:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:09:25.315813 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8040aa1f-b978-4ce3-a00a-ee8936bf1a63 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:25.353633 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:25.437898 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8040aa1f-b978-4ce3-a00a-ee8936bf1a63 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:25.486194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb756207-5e56-4029-a503-7048b5a69e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:25.486812 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb756207-5e56-4029-a503-7048b5a69e00 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:25.487086 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb756207-5e56-4029-a503-7048b5a69e00 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:25.487351 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb756207-5e56-4029-a503-7048b5a69e00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:25.494677 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb756207-5e56-4029-a503-7048b5a69e00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:25.496124 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb756207-5e56-4029-a503-7048b5a69e00 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:25.507600 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-bb756207-5e56-4029-a503-7048b5a69e00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:09:25.507600 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:09:25.507600 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:09:25.507600 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:09:25.507600 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:09:25.507600 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:09:25.507600 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:09:25.507600 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:09:25.507600 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:09:25.507600 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:09:25.507600 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:09:25.507600 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:09:25.507600 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 05:09:25.507600 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 05:09:25.507600 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 05:09:25.508628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 05:09:25.508628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:09:25.508628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:09:25.508628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:09:25.508628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:09:25.508628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:09:25.508628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:09:25.508628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:09:25.508628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:09:25.508628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:09:25.508628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:09:25.508628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:09:25.508628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:09:25.508628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:09:25.508628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:09:25.508628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:09:25.508628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:09:25.509749 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 05:09:25.509749 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 05:09:25.509749 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 05:09:25.509749 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 05:09:25.509749 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 05:09:25.509749 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 05:09:25.509749 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:09:25.509749 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:09:25.509749 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 645/1271] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:09:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:09:25.515427 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4e9ff-ec3e-40bc-bd2f-ad256bb9ea4d None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:25.584805 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:25.629539 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4e9ff-ec3e-40bc-bd2f-ad256bb9ea4d None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:25.667461 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c0a209-c80c-448a-9656-6dfa3a44476d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:25.668069 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c0a209-c80c-448a-9656-6dfa3a44476d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:25.668349 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c0a209-c80c-448a-9656-6dfa3a44476d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:25.668613 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c0a209-c80c-448a-9656-6dfa3a44476d None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:25.669308 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c0a209-c80c-448a-9656-6dfa3a44476d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:09:25.669612 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c0a209-c80c-448a-9656-6dfa3a44476d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:25.671087 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c0a209-c80c-448a-9656-6dfa3a44476d None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:25.683301 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 627/1272] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 05:09:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:09:25.689650 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb756207-5e56-4029-a503-7048b5a69e00 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:25.788289 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:25.803574 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb756207-5e56-4029-a503-7048b5a69e00 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:25.843081 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb008a76-c551-4ab7-bada-e73de760d11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:25.843716 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb008a76-c551-4ab7-bada-e73de760d11d None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:25.843987 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb008a76-c551-4ab7-bada-e73de760d11d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:25.844279 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb008a76-c551-4ab7-bada-e73de760d11d None admin] PATH_INFO: `/v3/projects/e01ce1b1282e44a39d659b87f120995e/users/bd4387035099416789009f0ecb8ad282/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:25.846385 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb008a76-c551-4ab7-bada-e73de760d11d None admin] RBAC: Authorizing `identity:create_grant(project_id=e01ce1b1282e44a39d659b87f120995e, user_id=bd4387035099416789009f0ecb8ad282, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:25.847811 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb008a76-c551-4ab7-bada-e73de760d11d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:25.867373 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 646/1273] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Aug 31 05:09:25 2022] PUT /identity/v3/projects/e01ce1b1282e44a39d659b87f120995e/users/bd4387035099416789009f0ecb8ad282/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:09:27.699610 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a16d0c-c368-493b-9ad6-5dbccc4e4bb2 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:27.700113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a16d0c-c368-493b-9ad6-5dbccc4e4bb2 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:27.700113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a16d0c-c368-493b-9ad6-5dbccc4e4bb2 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:27.701461 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 628/1274] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:09:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:09:27.707824 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89e93b-8db4-4339-87cf-ff255ed1c820 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:27.708521 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89e93b-8db4-4339-87cf-ff255ed1c820 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:27.708662 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89e93b-8db4-4339-87cf-ff255ed1c820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:27.733545 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-2f89e93b-8db4-4339-87cf-ff255ed1c820 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:09:27.734801 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2f89e93b-8db4-4339-87cf-ff255ed1c820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:09:27.811389 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 647/1275] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:09:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:09:28.073992 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a16d0c-c368-493b-9ad6-5dbccc4e4bb2 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:28.206737 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a16d0c-c368-493b-9ad6-5dbccc4e4bb2 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:28.226643 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:28.245091 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026a1fc-8895-4544-83c8-e080ae4ba39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:28.245720 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026a1fc-8895-4544-83c8-e080ae4ba39d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:28.245996 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026a1fc-8895-4544-83c8-e080ae4ba39d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:28.246296 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026a1fc-8895-4544-83c8-e080ae4ba39d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:28.258130 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026a1fc-8895-4544-83c8-e080ae4ba39d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 05:09:28.262815 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-3026a1fc-8895-4544-83c8-e080ae4ba39d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:09:28.262815 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:09:28.262815 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:09:28.262815 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:09:28.262815 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:09:28.262815 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:09:28.262815 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:09:28.262815 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:09:28.262815 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:09:28.262815 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:09:28.262815 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:09:28.262815 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:09:28.262815 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 05:09:28.262815 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 05:09:28.262815 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 05:09:28.262815 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 05:09:28.263985 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 05:09:28.263985 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 05:09:28.263985 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 05:09:28.263985 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 05:09:28.263985 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:09:28.263985 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:09:28.263985 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:09:28.263985 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:09:28.263985 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:09:28.263985 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:09:28.263985 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:09:28.263985 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:09:28.263985 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:09:28.263985 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:09:28.263985 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:09:28.263985 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:09:28.265062 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:09:28.265062 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:09:28.265062 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:09:28.265062 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:09:28.265062 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 05:09:28.265062 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 05:09:28.265062 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 05:09:28.265062 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 05:09:28.265062 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 05:09:28.265062 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 05:09:28.265062 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 05:09:28.265062 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:09:28.265062 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 629/1276] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Aug 31 05:09:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:09:28.270151 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89e93b-8db4-4339-87cf-ff255ed1c820 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:28.355280 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:28.379093 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89e93b-8db4-4339-87cf-ff255ed1c820 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:28.415540 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27a399f-c725-4512-aa6e-1312b7478fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:28.416132 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a399f-c725-4512-aa6e-1312b7478fd1 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:28.416403 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a399f-c725-4512-aa6e-1312b7478fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:28.416664 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a399f-c725-4512-aa6e-1312b7478fd1 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:28.417313 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27a399f-c725-4512-aa6e-1312b7478fd1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:09:28.417604 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27a399f-c725-4512-aa6e-1312b7478fd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:28.418999 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27a399f-c725-4512-aa6e-1312b7478fd1 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:28.427010 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 648/1277] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Aug 31 05:09:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:09:28.433053 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026a1fc-8895-4544-83c8-e080ae4ba39d None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:28.547348 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026a1fc-8895-4544-83c8-e080ae4ba39d None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:28.582299 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd1c4c-aa39-4c00-bef6-932fe138216d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:28.582905 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd1c4c-aa39-4c00-bef6-932fe138216d None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:28.583171 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd1c4c-aa39-4c00-bef6-932fe138216d None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:28.583433 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd1c4c-aa39-4c00-bef6-932fe138216d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:28.593696 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dd1c4c-aa39-4c00-bef6-932fe138216d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:28.595178 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dd1c4c-aa39-4c00-bef6-932fe138216d None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:28.609732 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-72dd1c4c-aa39-4c00-bef6-932fe138216d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:09:28.609732 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:09:28.609732 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:09:28.609732 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:09:28.609732 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:09:28.609732 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:09:28.609732 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:09:28.609732 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:09:28.609732 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:09:28.609732 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:09:28.609732 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:09:28.609732 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:09:28.609732 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:09:28.609732 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:09:28.609732 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:09:28.609732 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:09:28.610793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:09:28.610793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:09:28.610793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:09:28.610793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:09:28.610793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:09:28.610793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:09:28.610793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:09:28.610793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:09:28.610793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:09:28.610793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:09:28.610793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:09:28.610793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:09:28.610793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:09:28.610793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:09:28.610793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:09:28.610793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:09:28.610793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:09:28.612073 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:09:28.612073 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:09:28.612073 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:09:28.612073 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 630/1278] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:09:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:09:28.617004 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27a399f-c725-4512-aa6e-1312b7478fd1 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:28.727037 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27a399f-c725-4512-aa6e-1312b7478fd1 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:28.763646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cbef6f-a5be-4481-ba30-290f20fa33d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:28.764240 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbef6f-a5be-4481-ba30-290f20fa33d4 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:28.764511 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbef6f-a5be-4481-ba30-290f20fa33d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:28.764775 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbef6f-a5be-4481-ba30-290f20fa33d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:28.765405 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cbef6f-a5be-4481-ba30-290f20fa33d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:09:28.765695 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cbef6f-a5be-4481-ba30-290f20fa33d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:28.767105 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cbef6f-a5be-4481-ba30-290f20fa33d4 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:28.778753 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 649/1279] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:09:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:09:28.784562 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd1c4c-aa39-4c00-bef6-932fe138216d None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:28.815873 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:28.893815 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd1c4c-aa39-4c00-bef6-932fe138216d None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:28.930175 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992fa7fc-da45-4fdd-8863-f0c9d2fb5e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:28.930807 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992fa7fc-da45-4fdd-8863-f0c9d2fb5e24 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:28.931078 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992fa7fc-da45-4fdd-8863-f0c9d2fb5e24 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:28.931341 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992fa7fc-da45-4fdd-8863-f0c9d2fb5e24 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:28.938896 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992fa7fc-da45-4fdd-8863-f0c9d2fb5e24 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:28.940335 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992fa7fc-da45-4fdd-8863-f0c9d2fb5e24 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:28.951423 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-992fa7fc-da45-4fdd-8863-f0c9d2fb5e24 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 05:09:28.951423 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:09:28.951423 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:09:28.951423 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:09:28.951423 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:09:28.951423 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:09:28.951423 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:09:28.951423 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:09:28.951423 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:09:28.951423 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:09:28.951423 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:09:28.951423 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:09:28.951423 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 05:09:28.951423 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 05:09:28.951423 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 05:09:28.951423 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 05:09:28.952514 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:09:28.952514 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:09:28.952514 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 05:09:28.952514 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:09:28.952514 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 05:09:28.952514 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 05:09:28.952514 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:09:28.952514 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:09:28.952514 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:09:28.952514 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:09:28.952514 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:09:28.952514 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:09:28.952514 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:09:28.952514 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:09:28.952514 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:09:28.952514 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:09:28.952514 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:09:28.952514 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:09:28.953644 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:09:28.953644 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:09:28.953644 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 05:09:28.953644 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 05:09:28.953644 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 05:09:28.953644 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 05:09:28.953644 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 05:09:28.953644 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 05:09:28.953644 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 05:09:28.953644 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:09:28.953644 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 631/1280] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:09:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:09:28.958485 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cbef6f-a5be-4481-ba30-290f20fa33d4 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:28.964449 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:29.071096 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cbef6f-a5be-4481-ba30-290f20fa33d4 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:29.111765 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42f1f0-c02d-4fd5-81cb-d04d0ea3eef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:29.112359 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42f1f0-c02d-4fd5-81cb-d04d0ea3eef6 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:29.112630 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42f1f0-c02d-4fd5-81cb-d04d0ea3eef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:29.113214 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42f1f0-c02d-4fd5-81cb-d04d0ea3eef6 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:29.113936 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e42f1f0-c02d-4fd5-81cb-d04d0ea3eef6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:09:29.114269 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e42f1f0-c02d-4fd5-81cb-d04d0ea3eef6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:29.115744 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e42f1f0-c02d-4fd5-81cb-d04d0ea3eef6 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:29.132588 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 650/1281] 10.222.0.36 () {56 vars in 1143 bytes} [Wed Aug 31 05:09:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:09:29.138970 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992fa7fc-da45-4fdd-8863-f0c9d2fb5e24 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:29.238269 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:29.250394 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992fa7fc-da45-4fdd-8863-f0c9d2fb5e24 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:29.288827 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed594110-9e94-4508-844d-e4796bee7edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:29.289423 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed594110-9e94-4508-844d-e4796bee7edf None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:29.289695 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed594110-9e94-4508-844d-e4796bee7edf None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:29.289993 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed594110-9e94-4508-844d-e4796bee7edf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:29.357707 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed594110-9e94-4508-844d-e4796bee7edf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:29.359190 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed594110-9e94-4508-844d-e4796bee7edf None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:29.448544 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:29.460796 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 632/1282] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Aug 31 05:09:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 05:09:29.467017 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42f1f0-c02d-4fd5-81cb-d04d0ea3eef6 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:29.528278 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:29.573363 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42f1f0-c02d-4fd5-81cb-d04d0ea3eef6 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:29.608203 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cbe36f-0087-4bdf-8f1b-9b9b7452dd80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:29.608815 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cbe36f-0087-4bdf-8f1b-9b9b7452dd80 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:29.609089 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cbe36f-0087-4bdf-8f1b-9b9b7452dd80 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:29.609352 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cbe36f-0087-4bdf-8f1b-9b9b7452dd80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:29.619621 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cbe36f-0087-4bdf-8f1b-9b9b7452dd80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:29.621040 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cbe36f-0087-4bdf-8f1b-9b9b7452dd80 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:29.635346 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-d5cbe36f-0087-4bdf-8f1b-9b9b7452dd80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:09:29.635346 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:09:29.635346 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:09:29.635346 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:09:29.635346 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:09:29.635346 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:09:29.635346 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:09:29.635346 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:09:29.635346 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:09:29.635346 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:09:29.635346 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:09:29.635346 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:09:29.635346 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 05:09:29.635346 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 05:09:29.635346 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 05:09:29.635346 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 05:09:29.636404 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:09:29.636404 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:09:29.636404 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:09:29.636404 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:09:29.636404 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:09:29.636404 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:09:29.636404 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:09:29.636404 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:09:29.636404 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:09:29.636404 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:09:29.636404 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:09:29.636404 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:09:29.636404 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:09:29.636404 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:09:29.636404 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:09:29.636404 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:09:29.636404 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 05:09:29.637568 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 05:09:29.637568 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 05:09:29.637568 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:09:29.641118 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 651/1283] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:09:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:09:29.642044 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed594110-9e94-4508-844d-e4796bee7edf None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:29.744725 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed594110-9e94-4508-844d-e4796bee7edf None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:29.779638 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28cfef5-f310-4bcf-95ae-6608ed4b6bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:29.780169 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28cfef5-f310-4bcf-95ae-6608ed4b6bf1 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:29.780437 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28cfef5-f310-4bcf-95ae-6608ed4b6bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:29.780698 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28cfef5-f310-4bcf-95ae-6608ed4b6bf1 None admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:29.781327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28cfef5-f310-4bcf-95ae-6608ed4b6bf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:09:29.781619 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28cfef5-f310-4bcf-95ae-6608ed4b6bf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:29.783002 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28cfef5-f310-4bcf-95ae-6608ed4b6bf1 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:29.794591 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 633/1284] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Aug 31 05:09:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:09:29.800357 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cbe36f-0087-4bdf-8f1b-9b9b7452dd80 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:29.900715 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cbe36f-0087-4bdf-8f1b-9b9b7452dd80 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:29.914059 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:29.940803 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d7a237-a9f5-40a6-8e34-80e8aa2460e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:29.941403 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7a237-a9f5-40a6-8e34-80e8aa2460e6 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:29.941673 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7a237-a9f5-40a6-8e34-80e8aa2460e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:29.941934 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7a237-a9f5-40a6-8e34-80e8aa2460e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:29.948683 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d7a237-a9f5-40a6-8e34-80e8aa2460e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:29.950136 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d7a237-a9f5-40a6-8e34-80e8aa2460e6 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:29.961336 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-47d7a237-a9f5-40a6-8e34-80e8aa2460e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:09:29.961336 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:09:29.961336 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:09:29.961336 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:09:29.961336 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:09:29.961336 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:09:29.961336 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:09:29.961336 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:09:29.961336 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:09:29.961336 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:09:29.961336 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:09:29.961336 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:09:29.961336 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 05:09:29.961336 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 05:09:29.961336 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 05:09:29.962413 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 05:09:29.962413 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:09:29.962413 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:09:29.962413 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:09:29.962413 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:09:29.962413 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:09:29.962413 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:09:29.962413 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:09:29.962413 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 31 05:09:29.962413 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:09:29.962413 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:09:29.962413 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:09:29.962413 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:09:29.962413 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:09:29.962413 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:09:29.962413 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:09:29.962413 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:09:29.963526 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 05:09:29.963526 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 05:09:29.963526 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 05:09:29.963526 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 05:09:29.963526 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 05:09:29.963526 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 05:09:29.963526 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 05:09:29.963526 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 31 05:09:29.966181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 652/1285] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Aug 31 05:09:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:09:29.968636 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28cfef5-f310-4bcf-95ae-6608ed4b6bf1 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:30.029137 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:30.080908 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28cfef5-f310-4bcf-95ae-6608ed4b6bf1 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:30.116618 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6aa734-1f03-4cdd-ab7e-aa5993b88e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:30.117212 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6aa734-1f03-4cdd-ab7e-aa5993b88e6e None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:30.117484 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6aa734-1f03-4cdd-ab7e-aa5993b88e6e None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:30.117750 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6aa734-1f03-4cdd-ab7e-aa5993b88e6e None admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:30.118498 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6aa734-1f03-4cdd-ab7e-aa5993b88e6e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:09:30.119024 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6aa734-1f03-4cdd-ab7e-aa5993b88e6e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:30.121412 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6aa734-1f03-4cdd-ab7e-aa5993b88e6e None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:30.136540 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 634/1286] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Aug 31 05:09:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:09:30.146763 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d7a237-a9f5-40a6-8e34-80e8aa2460e6 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:30.255720 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d7a237-a9f5-40a6-8e34-80e8aa2460e6 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:30.289548 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747a5505-bd66-440a-880a-2d31b9d6fd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:30.290160 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a5505-bd66-440a-880a-2d31b9d6fd8f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:30.290429 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a5505-bd66-440a-880a-2d31b9d6fd8f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:30.290691 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a5505-bd66-440a-880a-2d31b9d6fd8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:30.291405 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747a5505-bd66-440a-880a-2d31b9d6fd8f None admin] RBAC: Adding query filter params (role.id=c3d5589dd2864be9a0f26f5802d874cd, scope.project.id=e01ce1b1282e44a39d659b87f120995e, user.id=bd4387035099416789009f0ecb8ad282) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:09:30.291700 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747a5505-bd66-440a-880a-2d31b9d6fd8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:30.293122 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747a5505-bd66-440a-880a-2d31b9d6fd8f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:30.298940 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 653/1287] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Aug 31 05:09:30 2022] GET /identity/v3/role_assignments?user.id=bd4387035099416789009f0ecb8ad282&scope.project.id=e01ce1b1282e44a39d659b87f120995e&role.id=c3d5589dd2864be9a0f26f5802d874cd => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:09:32.209548 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a598d02-bdc8-4e56-86f8-b5a933285939 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:32.210507 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a598d02-bdc8-4e56-86f8-b5a933285939 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:32.210507 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a598d02-bdc8-4e56-86f8-b5a933285939 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:32.212619 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 635/1288] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:09:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:09:32.218866 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9bc8d0-f538-4085-962a-ebfe6ada041f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:32.219593 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9bc8d0-f538-4085-962a-ebfe6ada041f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:32.219771 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9bc8d0-f538-4085-962a-ebfe6ada041f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:32.247324 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ae9bc8d0-f538-4085-962a-ebfe6ada041f None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:09:32.248511 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ae9bc8d0-f538-4085-962a-ebfe6ada041f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:09:32.279225 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:32.319702 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 654/1289] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:09:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:09:32.582262 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a598d02-bdc8-4e56-86f8-b5a933285939 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:32.700256 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a598d02-bdc8-4e56-86f8-b5a933285939 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:32.711020 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:32.736934 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3964964b-5139-4135-a2e6-0b2f880760ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:32.737859 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964964b-5139-4135-a2e6-0b2f880760ab None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:32.738172 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964964b-5139-4135-a2e6-0b2f880760ab None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:32.738435 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964964b-5139-4135-a2e6-0b2f880760ab None admin] PATH_INFO: `/v3/services/placement` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:32.739008 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3964964b-5139-4135-a2e6-0b2f880760ab None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:32.740506 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3964964b-5139-4135-a2e6-0b2f880760ab None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:32.749245 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-3964964b-5139-4135-a2e6-0b2f880760ab None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 05:09:32.749245 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:09:32.749245 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 05:09:32.749245 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 05:09:32.749245 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 05:09:32.749245 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 05:09:32.749245 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 05:09:32.749245 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:09:32.749245 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 05:09:32.749245 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:09:32.749245 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 05:09:32.749245 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:09:32.749245 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:09:32.749245 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:09:32.749245 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:09:32.749245 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:09:32.749245 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:09:32.750485 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:09:32.750485 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:09:32.750485 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:09:32.750485 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:09:32.750485 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:09:32.750485 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:09:32.750485 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 05:09:32.750485 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 05:09:32.750485 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 05:09:32.750485 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 05:09:32.750485 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:09:32.750485 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:09:32.750485 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:09:32.750485 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:09:32.750485 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:09:32.750485 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:09:32.751595 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:09:32.751595 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:09:32.751595 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:09:32.751595 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:09:32.751595 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:09:32.751595 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:09:32.751595 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:09:32.751595 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:09:32.751595 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:09:32.751595 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:09:32.751595 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 05:09:32.751595 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:09:32.751595 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 05:09:32.751595 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:09:32.751595 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 636/1290] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Aug 31 05:09:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:09:32.757933 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9bc8d0-f538-4085-962a-ebfe6ada041f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:32.861661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9bc8d0-f538-4085-962a-ebfe6ada041f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:32.898923 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79cbcc9-e831-4943-aa0b-39a9be31d6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:32.899513 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79cbcc9-e831-4943-aa0b-39a9be31d6d9 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:32.899782 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79cbcc9-e831-4943-aa0b-39a9be31d6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:32.900045 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79cbcc9-e831-4943-aa0b-39a9be31d6d9 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:32.900674 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79cbcc9-e831-4943-aa0b-39a9be31d6d9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:09:32.900975 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79cbcc9-e831-4943-aa0b-39a9be31d6d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:32.902356 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79cbcc9-e831-4943-aa0b-39a9be31d6d9 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:32.907496 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 655/1291] 10.222.0.36 () {56 vars in 1116 bytes} [Wed Aug 31 05:09:32 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:09:32.913660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3964964b-5139-4135-a2e6-0b2f880760ab None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:33.020078 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3964964b-5139-4135-a2e6-0b2f880760ab None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:33.055945 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab7abe0-1b1d-4e5c-aa5b-d7e175fdb8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:33.056538 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab7abe0-1b1d-4e5c-aa5b-d7e175fdb8e9 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:33.056809 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab7abe0-1b1d-4e5c-aa5b-d7e175fdb8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:33.057074 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab7abe0-1b1d-4e5c-aa5b-d7e175fdb8e9 None admin] PATH_INFO: `/v3/services` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:33.057695 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab7abe0-1b1d-4e5c-aa5b-d7e175fdb8e9 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:09:33.057990 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab7abe0-1b1d-4e5c-aa5b-d7e175fdb8e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:33.059388 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab7abe0-1b1d-4e5c-aa5b-d7e175fdb8e9 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:33.063829 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 637/1292] 10.222.0.36 () {56 vars in 1116 bytes} [Wed Aug 31 05:09:32 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:09:34.931261 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ee627-b239-42c3-9837-263edb90db3d None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:34.932060 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ee627-b239-42c3-9837-263edb90db3d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:34.932157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ee627-b239-42c3-9837-263edb90db3d None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:34.935426 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 656/1293] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:09:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:09:34.944042 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db243e-8aa0-4d54-9fa3-e08cc60adc96 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:34.944509 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db243e-8aa0-4d54-9fa3-e08cc60adc96 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:34.944607 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db243e-8aa0-4d54-9fa3-e08cc60adc96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:34.966824 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-e7db243e-8aa0-4d54-9fa3-e08cc60adc96 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:09:34.968029 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-e7db243e-8aa0-4d54-9fa3-e08cc60adc96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:09:35.001226 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:35.042603 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 638/1294] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:09:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:09:35.308342 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266ee627-b239-42c3-9837-263edb90db3d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:35.352289 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:35.426936 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266ee627-b239-42c3-9837-263edb90db3d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:35.463410 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd76579-433d-4551-b573-d61411279d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:35.464258 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd76579-433d-4551-b573-d61411279d5b None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:35.464618 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd76579-433d-4551-b573-d61411279d5b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:35.464967 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd76579-433d-4551-b573-d61411279d5b None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:35.465926 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd76579-433d-4551-b573-d61411279d5b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:35.468200 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd76579-433d-4551-b573-d61411279d5b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:35.488248 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 657/1295] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Aug 31 05:09:35 2022] POST /identity/v3/services => generated 246 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:09:37.391733 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d9a36-c179-4196-b433-735c7396e730 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:37.392312 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d9a36-c179-4196-b433-735c7396e730 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:37.392312 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d9a36-c179-4196-b433-735c7396e730 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:37.394500 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 639/1296] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:09:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:09:37.401620 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637785b4-e2c1-4a2b-bc33-ef99bfe90851 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:37.402047 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637785b4-e2c1-4a2b-bc33-ef99bfe90851 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:37.402378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637785b4-e2c1-4a2b-bc33-ef99bfe90851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:37.431240 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-637785b4-e2c1-4a2b-bc33-ef99bfe90851 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:09:37.432416 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-637785b4-e2c1-4a2b-bc33-ef99bfe90851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:09:37.508026 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 658/1297] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:09:37 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:09:37.777565 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d9a36-c179-4196-b433-735c7396e730 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:37.886906 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d9a36-c179-4196-b433-735c7396e730 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:37.922721 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dca3a5f-8fbb-4136-a18e-780928cf4b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:37.923279 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca3a5f-8fbb-4136-a18e-780928cf4b80 None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:37.923552 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca3a5f-8fbb-4136-a18e-780928cf4b80 None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:37.923817 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca3a5f-8fbb-4136-a18e-780928cf4b80 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:37.924398 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dca3a5f-8fbb-4136-a18e-780928cf4b80 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:37.925787 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dca3a5f-8fbb-4136-a18e-780928cf4b80 None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:37.936327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-2dca3a5f-8fbb-4136-a18e-780928cf4b80 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 05:09:37.936327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:09:37.936327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 05:09:37.936327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 05:09:37.936327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 05:09:37.936327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 05:09:37.936327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 05:09:37.936327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:09:37.936327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 05:09:37.936327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:09:37.936327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 05:09:37.936327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:09:37.936327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:09:37.936327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:09:37.936327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:09:37.936327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:09:37.936327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:09:37.937614 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:09:37.937614 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:09:37.937614 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:09:37.937614 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:09:37.937614 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:09:37.937614 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:09:37.937614 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 05:09:37.937614 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 05:09:37.937614 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 05:09:37.937614 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 05:09:37.937614 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:09:37.937614 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:09:37.937614 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:09:37.937614 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:09:37.937614 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:09:37.937614 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:09:37.938658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:09:37.938658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:09:37.938658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:09:37.938658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:09:37.938658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:09:37.938658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:09:37.938658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:09:37.938658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:09:37.938658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:09:37.938658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:09:37.938658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 05:09:37.938658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:09:37.938658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 05:09:37.938658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:09:37.938658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 640/1298] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Aug 31 05:09:37 2022] GET /identity/v3/services/placement => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:09:37.945860 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637785b4-e2c1-4a2b-bc33-ef99bfe90851 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:37.968674 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:38.057723 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637785b4-e2c1-4a2b-bc33-ef99bfe90851 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:38.092568 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa34e8-4256-4576-a5b9-0fa1a243e9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:38.093161 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa34e8-4256-4576-a5b9-0fa1a243e9ff None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:38.093427 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa34e8-4256-4576-a5b9-0fa1a243e9ff None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:38.093687 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa34e8-4256-4576-a5b9-0fa1a243e9ff None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:38.094315 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fa34e8-4256-4576-a5b9-0fa1a243e9ff None admin] RBAC: Adding query filter params (name=placement) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:09:38.094610 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fa34e8-4256-4576-a5b9-0fa1a243e9ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:38.095967 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fa34e8-4256-4576-a5b9-0fa1a243e9ff None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:38.101012 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 659/1299] 10.222.0.36 () {56 vars in 1116 bytes} [Wed Aug 31 05:09:37 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:09:38.110726 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dca3a5f-8fbb-4136-a18e-780928cf4b80 None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:38.149573 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:38.220140 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dca3a5f-8fbb-4136-a18e-780928cf4b80 None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:38.262990 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d86e2e-c2ae-4b3b-879c-0df1d83e81fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:38.263587 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d86e2e-c2ae-4b3b-879c-0df1d83e81fe None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:38.263860 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d86e2e-c2ae-4b3b-879c-0df1d83e81fe None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:38.264121 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d86e2e-c2ae-4b3b-879c-0df1d83e81fe None admin] PATH_INFO: `/v3/endpoints` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:38.264810 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d86e2e-c2ae-4b3b-879c-0df1d83e81fe None admin] RBAC: Adding query filter params (interface=public, service_id=8b52acbae11b4f179fc9532bc31f0665, region_id=RegionOne) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:09:38.265116 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d86e2e-c2ae-4b3b-879c-0df1d83e81fe None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:38.266526 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d86e2e-c2ae-4b3b-879c-0df1d83e81fe None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:38.271746 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 641/1300] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Aug 31 05:09:38 2022] GET /identity/v3/endpoints?service_id=8b52acbae11b4f179fc9532bc31f0665&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:09:38.277904 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa34e8-4256-4576-a5b9-0fa1a243e9ff None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:38.370325 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:38.393974 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa34e8-4256-4576-a5b9-0fa1a243e9ff None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:38.428914 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b82c1-8491-492f-ab70-093ebcefad10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:38.429491 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b82c1-8491-492f-ab70-093ebcefad10 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:38.429768 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b82c1-8491-492f-ab70-093ebcefad10 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:38.430104 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b82c1-8491-492f-ab70-093ebcefad10 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:38.430646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607b82c1-8491-492f-ab70-093ebcefad10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:38.432000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607b82c1-8491-492f-ab70-093ebcefad10 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:38.438703 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 660/1301] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Aug 31 05:09:38 2022] GET /identity/v3/services => generated 2191 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:09:40.312274 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec681a5f-7f91-4db2-bd07-13f0a1018bd1 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:40.313189 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec681a5f-7f91-4db2-bd07-13f0a1018bd1 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:40.313189 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec681a5f-7f91-4db2-bd07-13f0a1018bd1 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:40.315298 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 642/1302] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:09:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:09:40.320932 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43224bc9-b253-4847-a80b-2cc518def36d None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:40.321364 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43224bc9-b253-4847-a80b-2cc518def36d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:40.321471 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43224bc9-b253-4847-a80b-2cc518def36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:40.340941 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-43224bc9-b253-4847-a80b-2cc518def36d None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:09:40.342161 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-43224bc9-b253-4847-a80b-2cc518def36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:09:40.412568 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 661/1303] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:09:40 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 05:09:40.680655 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec681a5f-7f91-4db2-bd07-13f0a1018bd1 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:40.784880 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:40.796616 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec681a5f-7f91-4db2-bd07-13f0a1018bd1 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:40.841948 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b769c4bd-37a1-4ca0-838c-2fda2afc199c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:40.842595 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769c4bd-37a1-4ca0-838c-2fda2afc199c None admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:40.842871 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769c4bd-37a1-4ca0-838c-2fda2afc199c None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:40.843146 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769c4bd-37a1-4ca0-838c-2fda2afc199c None admin] PATH_INFO: `/v3/services/placement` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:40.843723 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b769c4bd-37a1-4ca0-838c-2fda2afc199c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:40.845106 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b769c4bd-37a1-4ca0-838c-2fda2afc199c None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:40.855124 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application [None req-b769c4bd-37a1-4ca0-838c-2fda2afc199c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 05:09:40.855124 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:09:40.855124 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 05:09:40.855124 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 05:09:40.855124 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 05:09:40.855124 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 05:09:40.855124 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 05:09:40.855124 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:09:40.855124 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 05:09:40.855124 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:09:40.855124 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 05:09:40.855124 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:09:40.855124 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 05:09:40.855124 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 05:09:40.855124 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 05:09:40.855124 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 05:09:40.855124 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 05:09:40.856248 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 05:09:40.856248 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 05:09:40.856248 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 05:09:40.856248 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 05:09:40.856248 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 05:09:40.856248 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 05:09:40.856248 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 05:09:40.856248 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 05:09:40.856248 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 05:09:40.856248 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 05:09:40.856248 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 05:09:40.856248 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 05:09:40.856248 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 05:09:40.856248 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 05:09:40.856248 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 05:09:40.856248 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 05:09:40.857255 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 05:09:40.857255 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application with Lock( Aug 31 05:09:40.857255 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 05:09:40.857255 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self._enter() Aug 31 05:09:40.857255 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 05:09:40.857255 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 05:09:40.857255 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 05:09:40.857255 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application return self.creator() Aug 31 05:09:40.857255 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 05:09:40.857255 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application created_value = creator( Aug 31 05:09:40.857255 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 05:09:40.857255 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 05:09:40.857255 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 05:09:40.857255 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: ERROR keystone.server.flask.application Aug 31 05:09:40.857255 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 643/1304] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Aug 31 05:09:40 2022] GET /identity/v3/services/placement => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 05:09:40.863051 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43224bc9-b253-4847-a80b-2cc518def36d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:40.964466 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:40.988735 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43224bc9-b253-4847-a80b-2cc518def36d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:41.023191 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e75ac1-a06d-419e-bafd-146957795581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:41.023792 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e75ac1-a06d-419e-bafd-146957795581 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:41.024064 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e75ac1-a06d-419e-bafd-146957795581 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:41.024330 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e75ac1-a06d-419e-bafd-146957795581 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:41.024946 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e75ac1-a06d-419e-bafd-146957795581 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:09:41.025240 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e75ac1-a06d-419e-bafd-146957795581 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:41.026661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e75ac1-a06d-419e-bafd-146957795581 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:41.032176 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 662/1305] 10.222.0.36 () {56 vars in 1116 bytes} [Wed Aug 31 05:09:40 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:09:41.038323 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b769c4bd-37a1-4ca0-838c-2fda2afc199c None admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:09:41.163106 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b769c4bd-37a1-4ca0-838c-2fda2afc199c None admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:09:41.201478 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce3154-9665-489a-97c8-276bb8ec340a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:09:41.202110 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce3154-9665-489a-97c8-276bb8ec340a None admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:09:41.202389 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce3154-9665-489a-97c8-276bb8ec340a None admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:09:41.202658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce3154-9665-489a-97c8-276bb8ec340a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:09:41.203348 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce3154-9665-489a-97c8-276bb8ec340a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:09:41.204760 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce3154-9665-489a-97c8-276bb8ec340a None admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:09:41.232722 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 644/1306] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:09:41 2022] POST /identity/v3/endpoints => generated 331 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:09:51.029213 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=30 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:09:51.225007 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:10:44.308720 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d17e0-ccea-4345-983d-35b066906c39 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:10:44.309388 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d17e0-ccea-4345-983d-35b066906c39 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:10:44.309388 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d17e0-ccea-4345-983d-35b066906c39 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:10:44.314421 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 663/1307] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:10:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:10:44.315927 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d31a278-3485-4bb9-ba25-768ae5e3cdbb None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:10:44.316339 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d31a278-3485-4bb9-ba25-768ae5e3cdbb None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:10:44.316559 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d31a278-3485-4bb9-ba25-768ae5e3cdbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:10:44.392725 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-9d31a278-3485-4bb9-ba25-768ae5e3cdbb None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:10:44.393854 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-9d31a278-3485-4bb9-ba25-768ae5e3cdbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:10:44.522593 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 645/1308] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:10:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:10:44.792941 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759eaf36-0b2c-4864-b6e8-90dde5628def None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:10:44.793295 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759eaf36-0b2c-4864-b6e8-90dde5628def None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:10:44.793771 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759eaf36-0b2c-4864-b6e8-90dde5628def None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:10:44.795452 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 664/1309] 10.222.0.36 () {54 vars in 965 bytes} [Wed Aug 31 05:10:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:10:44.804051 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79462caa-ac0f-46d2-872a-cb071262e5e0 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:10:44.804357 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79462caa-ac0f-46d2-872a-cb071262e5e0 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:10:44.804623 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79462caa-ac0f-46d2-872a-cb071262e5e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:10:44.893363 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-79462caa-ac0f-46d2-872a-cb071262e5e0 None None] MFA Rules not processed for user `090e1409506542f6831e1f65d44aae74`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:10:44.894693 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-79462caa-ac0f-46d2-872a-cb071262e5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:10:44.918270 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:10:44.980691 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 646/1310] 10.222.0.36 () {58 vars in 1062 bytes} [Wed Aug 31 05:10:44 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:10:44.990640 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759eaf36-0b2c-4864-b6e8-90dde5628def None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:10:44.993556 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-759eaf36-0b2c-4864-b6e8-90dde5628def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:10:45.114975 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759eaf36-0b2c-4864-b6e8-90dde5628def None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:10:45.189315 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac5d96b-4d66-4163-b407-64cf39b9e109 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:10:45.189843 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac5d96b-4d66-4163-b407-64cf39b9e109 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:10:45.190143 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac5d96b-4d66-4163-b407-64cf39b9e109 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:10:45.190407 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac5d96b-4d66-4163-b407-64cf39b9e109 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:10:45.191978 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0ac5d96b-4d66-4163-b407-64cf39b9e109 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:10:45.246974 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:10:45.306279 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac5d96b-4d66-4163-b407-64cf39b9e109 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:10:45.307751 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac5d96b-4d66-4163-b407-64cf39b9e109 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:10:45.333236 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 665/1311] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:10:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:10:47.482452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38476f4d-5a0c-40b1-ab1d-0a47333013a8 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:10:47.482985 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38476f4d-5a0c-40b1-ab1d-0a47333013a8 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:10:47.482985 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38476f4d-5a0c-40b1-ab1d-0a47333013a8 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:10:47.484691 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 647/1312] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:10:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:10:47.490176 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb85a1-087a-4c1e-b206-65ee98935490 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:10:47.490648 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb85a1-087a-4c1e-b206-65ee98935490 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:10:47.490744 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb85a1-087a-4c1e-b206-65ee98935490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:10:47.576230 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-20fb85a1-087a-4c1e-b206-65ee98935490 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:10:47.577807 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-20fb85a1-087a-4c1e-b206-65ee98935490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:10:47.691976 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 666/1313] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:10:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:10:47.966689 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38476f4d-5a0c-40b1-ab1d-0a47333013a8 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:10:47.968220 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-38476f4d-5a0c-40b1-ab1d-0a47333013a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:10:48.070272 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38476f4d-5a0c-40b1-ab1d-0a47333013a8 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:10:48.146495 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:10:48.149423 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e1ecb-2642-49b5-b502-624b2cfdb3d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:10:48.150104 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e1ecb-2642-49b5-b502-624b2cfdb3d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:10:48.150768 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e1ecb-2642-49b5-b502-624b2cfdb3d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:10:48.151115 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e1ecb-2642-49b5-b502-624b2cfdb3d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:10:48.153301 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-d50e1ecb-2642-49b5-b502-624b2cfdb3d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:10:48.268704 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50e1ecb-2642-49b5-b502-624b2cfdb3d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:10:48.270203 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50e1ecb-2642-49b5-b502-624b2cfdb3d4 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:10:48.289662 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 648/1314] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:10:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:10:50.494912 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131f9fa-b6f4-4864-a50c-fdd451ea4496 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:10:50.495499 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131f9fa-b6f4-4864-a50c-fdd451ea4496 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:10:50.495499 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131f9fa-b6f4-4864-a50c-fdd451ea4496 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:10:50.496794 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 667/1315] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:10:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:10:50.502113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42c578-f12f-4e6c-964a-bab651bb39f8 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:10:50.502558 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42c578-f12f-4e6c-964a-bab651bb39f8 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:10:50.502664 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42c578-f12f-4e6c-964a-bab651bb39f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:10:50.575482 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-de42c578-f12f-4e6c-964a-bab651bb39f8 None None] MFA Rules not processed for user `bbcee328f8db4f5cb50f75830eb1cb5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:10:50.576654 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-de42c578-f12f-4e6c-964a-bab651bb39f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:10:50.632575 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:10:50.681565 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 649/1316] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:10:50 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:10:50.963145 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2131f9fa-b6f4-4864-a50c-fdd451ea4496 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:10:50.965604 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2131f9fa-b6f4-4864-a50c-fdd451ea4496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:10:50.977440 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:10:51.089741 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2131f9fa-b6f4-4864-a50c-fdd451ea4496 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:10:51.179273 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c226bbc-021b-4c2d-a439-5f2d40957eae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:10:51.179810 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c226bbc-021b-4c2d-a439-5f2d40957eae service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:10:51.180085 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c226bbc-021b-4c2d-a439-5f2d40957eae service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:10:51.180351 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c226bbc-021b-4c2d-a439-5f2d40957eae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:10:51.181923 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6c226bbc-021b-4c2d-a439-5f2d40957eae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:10:51.245518 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:10:51.296388 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c226bbc-021b-4c2d-a439-5f2d40957eae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:10:51.297899 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c226bbc-021b-4c2d-a439-5f2d40957eae service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:10:51.316691 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 668/1317] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:10:50 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:10:53.761208 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af85c283-a7e0-4e60-ba0a-a4831ddbee86 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:10:53.762039 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af85c283-a7e0-4e60-ba0a-a4831ddbee86 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:10:53.762039 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af85c283-a7e0-4e60-ba0a-a4831ddbee86 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:10:53.764172 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 650/1318] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:10:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:10:53.770462 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b0c5ce-b2a2-4c8c-bcd1-124b30d7b732 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:10:53.770876 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b0c5ce-b2a2-4c8c-bcd1-124b30d7b732 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:10:53.770996 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b0c5ce-b2a2-4c8c-bcd1-124b30d7b732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:10:53.846660 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f3b0c5ce-b2a2-4c8c-bcd1-124b30d7b732 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:10:53.847860 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f3b0c5ce-b2a2-4c8c-bcd1-124b30d7b732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:10:53.954185 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 669/1319] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:10:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:10:54.233980 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af85c283-a7e0-4e60-ba0a-a4831ddbee86 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:10:54.236262 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-af85c283-a7e0-4e60-ba0a-a4831ddbee86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:10:54.355067 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af85c283-a7e0-4e60-ba0a-a4831ddbee86 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:10:54.436664 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e6881-efba-4cc8-995b-0da4601a8198 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:10:54.437257 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e6881-efba-4cc8-995b-0da4601a8198 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:10:54.437531 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e6881-efba-4cc8-995b-0da4601a8198 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:10:54.437788 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e6881-efba-4cc8-995b-0da4601a8198 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:10:54.439402 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-5d9e6881-efba-4cc8-995b-0da4601a8198 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:10:54.446701 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:10:54.554104 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9e6881-efba-4cc8-995b-0da4601a8198 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:10:54.555547 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9e6881-efba-4cc8-995b-0da4601a8198 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:10:54.575121 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 651/1320] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:10:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:10:57.117450 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0398c2-39e4-4c85-8f41-d4be1e897600 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:10:57.117991 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0398c2-39e4-4c85-8f41-d4be1e897600 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:10:57.117991 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0398c2-39e4-4c85-8f41-d4be1e897600 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:10:57.119564 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 670/1321] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:10:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:10:57.125019 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f00916-4b0b-4dcb-b883-22a150b92f6b None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:10:57.125588 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f00916-4b0b-4dcb-b883-22a150b92f6b None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:10:57.125685 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f00916-4b0b-4dcb-b883-22a150b92f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:10:57.193682 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-71f00916-4b0b-4dcb-b883-22a150b92f6b None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:10:57.194911 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-71f00916-4b0b-4dcb-b883-22a150b92f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:10:57.253638 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:10:57.293492 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 652/1322] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:10:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:10:57.590699 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06a80d-ae7f-4509-bfe7-71d375d57cc4 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:10:57.590960 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06a80d-ae7f-4509-bfe7-71d375d57cc4 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:10:57.591230 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06a80d-ae7f-4509-bfe7-71d375d57cc4 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:10:57.592573 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 671/1323] 10.222.0.36 () {54 vars in 965 bytes} [Wed Aug 31 05:10:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:10:57.599071 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030d09f-f57b-415a-83de-dda698822fc7 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:10:57.599434 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030d09f-f57b-415a-83de-dda698822fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:10:57.599774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030d09f-f57b-415a-83de-dda698822fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:10:57.682036 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-b030d09f-f57b-415a-83de-dda698822fc7 None None] MFA Rules not processed for user `090e1409506542f6831e1f65d44aae74`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:10:57.683273 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b030d09f-f57b-415a-83de-dda698822fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:10:57.763767 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 653/1324] 10.222.0.36 () {58 vars in 1062 bytes} [Wed Aug 31 05:10:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:10:57.773087 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d06a80d-ae7f-4509-bfe7-71d375d57cc4 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:10:57.774622 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1d06a80d-ae7f-4509-bfe7-71d375d57cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:10:57.795338 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:10:57.884295 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d06a80d-ae7f-4509-bfe7-71d375d57cc4 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:10:57.961115 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85627c99-111e-476e-9ed5-21a4f4d55ffd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:10:57.961726 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85627c99-111e-476e-9ed5-21a4f4d55ffd service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:10:57.962023 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85627c99-111e-476e-9ed5-21a4f4d55ffd service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:10:57.962296 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85627c99-111e-476e-9ed5-21a4f4d55ffd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:10:57.963894 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-85627c99-111e-476e-9ed5-21a4f4d55ffd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:10:58.037293 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:10:58.077988 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85627c99-111e-476e-9ed5-21a4f4d55ffd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:10:58.079535 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85627c99-111e-476e-9ed5-21a4f4d55ffd service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:10:58.097687 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 672/1325] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:10:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:11:00.265774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf4e95-f1d2-44b9-bf0a-2b3ad4d97f41 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:00.266474 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf4e95-f1d2-44b9-bf0a-2b3ad4d97f41 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:00.266474 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf4e95-f1d2-44b9-bf0a-2b3ad4d97f41 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:00.267765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 654/1326] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:11:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:11:00.273334 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ade61-0a14-413b-adf7-bfa43acbe6aa None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:00.273826 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ade61-0a14-413b-adf7-bfa43acbe6aa None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:00.273826 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ade61-0a14-413b-adf7-bfa43acbe6aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:00.341485 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ef1ade61-0a14-413b-adf7-bfa43acbe6aa None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:11:00.342722 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ef1ade61-0a14-413b-adf7-bfa43acbe6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:00.458793 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 673/1327] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:11:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:11:00.739606 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf4e95-f1d2-44b9-bf0a-2b3ad4d97f41 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:11:00.741191 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-01bf4e95-f1d2-44b9-bf0a-2b3ad4d97f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:00.804037 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:11:00.851220 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf4e95-f1d2-44b9-bf0a-2b3ad4d97f41 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:11:00.927392 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b8ae8-51cd-45e4-b2fd-3419bb1c5652 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:11:00.927897 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b8ae8-51cd-45e4-b2fd-3419bb1c5652 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:00.928164 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b8ae8-51cd-45e4-b2fd-3419bb1c5652 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:00.928425 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b8ae8-51cd-45e4-b2fd-3419bb1c5652 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:00.930027 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-1c7b8ae8-51cd-45e4-b2fd-3419bb1c5652 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:01.038213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:11:01.040644 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b8ae8-51cd-45e4-b2fd-3419bb1c5652 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:11:01.042455 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b8ae8-51cd-45e4-b2fd-3419bb1c5652 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:11:01.060865 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 655/1328] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:11:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:11:02.975336 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54222872-d0c4-4002-b4c6-683cc6b46149 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:02.975797 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54222872-d0c4-4002-b4c6-683cc6b46149 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:02.975797 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54222872-d0c4-4002-b4c6-683cc6b46149 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:02.979011 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 674/1329] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:11:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:11:02.982685 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95459f-69e7-4143-ab59-3aa15de16d55 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:02.983124 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95459f-69e7-4143-ab59-3aa15de16d55 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:02.983210 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95459f-69e7-4143-ab59-3aa15de16d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:03.057518 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-1d95459f-69e7-4143-ab59-3aa15de16d55 None None] MFA Rules not processed for user `bbcee328f8db4f5cb50f75830eb1cb5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:11:03.058813 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-1d95459f-69e7-4143-ab59-3aa15de16d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:03.155463 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 656/1330] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:11:02 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:11:03.436241 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54222872-d0c4-4002-b4c6-683cc6b46149 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:11:03.437782 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-54222872-d0c4-4002-b4c6-683cc6b46149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:03.470318 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:11:03.546460 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54222872-d0c4-4002-b4c6-683cc6b46149 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:11:03.632998 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4633e09a-a15c-404d-aa33-8063d118c74a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:11:03.633622 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633e09a-a15c-404d-aa33-8063d118c74a service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:03.633897 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633e09a-a15c-404d-aa33-8063d118c74a service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:03.634202 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633e09a-a15c-404d-aa33-8063d118c74a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:03.635857 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4633e09a-a15c-404d-aa33-8063d118c74a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:03.719451 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:11:03.753284 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4633e09a-a15c-404d-aa33-8063d118c74a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:11:03.754815 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4633e09a-a15c-404d-aa33-8063d118c74a service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:11:03.774112 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 675/1331] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:11:03 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:11:05.879135 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea1fcc-2fbd-4b13-a2fb-7d41833b5548 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:05.880068 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea1fcc-2fbd-4b13-a2fb-7d41833b5548 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:05.880068 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea1fcc-2fbd-4b13-a2fb-7d41833b5548 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:05.882211 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 657/1332] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:11:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:11:05.887481 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7663f6dd-cb4b-4246-a4df-69f63e3cfbb0 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:05.887915 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7663f6dd-cb4b-4246-a4df-69f63e3cfbb0 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:05.888020 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7663f6dd-cb4b-4246-a4df-69f63e3cfbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:05.956267 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-7663f6dd-cb4b-4246-a4df-69f63e3cfbb0 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:11:05.957383 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7663f6dd-cb4b-4246-a4df-69f63e3cfbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:06.057024 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 676/1333] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:11:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:11:06.333687 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea1fcc-2fbd-4b13-a2fb-7d41833b5548 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:11:06.336186 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-35ea1fcc-2fbd-4b13-a2fb-7d41833b5548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:06.415701 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:11:06.451731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea1fcc-2fbd-4b13-a2fb-7d41833b5548 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:11:06.538054 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c68affd-61b3-4e65-b43c-791f9d17ff41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:11:06.538686 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68affd-61b3-4e65-b43c-791f9d17ff41 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:06.538965 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68affd-61b3-4e65-b43c-791f9d17ff41 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:06.539240 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68affd-61b3-4e65-b43c-791f9d17ff41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:06.540853 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-4c68affd-61b3-4e65-b43c-791f9d17ff41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:06.659810 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c68affd-61b3-4e65-b43c-791f9d17ff41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:11:06.661268 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c68affd-61b3-4e65-b43c-791f9d17ff41 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:11:06.671743 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:11:06.681711 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 658/1334] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:11:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:11:08.640408 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac39eac-bd50-40cf-8d53-241af5d5b024 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:08.641016 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac39eac-bd50-40cf-8d53-241af5d5b024 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:08.641016 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac39eac-bd50-40cf-8d53-241af5d5b024 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:08.642351 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 677/1335] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:11:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:11:08.651165 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100fbaa8-3080-493d-951c-5d3e4a13e605 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:08.651885 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100fbaa8-3080-493d-951c-5d3e4a13e605 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:08.652288 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100fbaa8-3080-493d-951c-5d3e4a13e605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:08.728798 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-100fbaa8-3080-493d-951c-5d3e4a13e605 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:11:08.730023 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-100fbaa8-3080-493d-951c-5d3e4a13e605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:08.843134 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 659/1336] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:11:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:11:09.157603 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac39eac-bd50-40cf-8d53-241af5d5b024 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:11:09.159205 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-cac39eac-bd50-40cf-8d53-241af5d5b024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:09.202351 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:11:09.274190 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac39eac-bd50-40cf-8d53-241af5d5b024 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:11:09.350293 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ce50e-7986-4109-bc8f-10789b0b7d76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:11:09.350898 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ce50e-7986-4109-bc8f-10789b0b7d76 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:09.351169 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ce50e-7986-4109-bc8f-10789b0b7d76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:09.351434 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ce50e-7986-4109-bc8f-10789b0b7d76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:09.353037 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-788ce50e-7986-4109-bc8f-10789b0b7d76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:09.447373 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:11:09.476009 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ce50e-7986-4109-bc8f-10789b0b7d76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:11:09.477559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ce50e-7986-4109-bc8f-10789b0b7d76 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:11:09.498654 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 678/1337] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:11:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:11:11.766353 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b147aab8-de94-4038-9cce-f2f5eeafc71f None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:11.767238 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b147aab8-de94-4038-9cce-f2f5eeafc71f None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:11.767238 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b147aab8-de94-4038-9cce-f2f5eeafc71f None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:11.769445 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 660/1338] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:11:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:11:11.776039 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e055a-3c34-4870-9d5e-5378fa602e68 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:11.776529 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e055a-3c34-4870-9d5e-5378fa602e68 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:11.776624 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e055a-3c34-4870-9d5e-5378fa602e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:11.846514 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-8f8e055a-3c34-4870-9d5e-5378fa602e68 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:11:11.847608 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8f8e055a-3c34-4870-9d5e-5378fa602e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:11.946577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 679/1339] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:11:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:11:12.224087 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b147aab8-de94-4038-9cce-f2f5eeafc71f None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:11:12.226602 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b147aab8-de94-4038-9cce-f2f5eeafc71f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:12.327048 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:11:12.357853 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b147aab8-de94-4038-9cce-f2f5eeafc71f None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:11:12.437100 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9d375f-334b-47f5-8681-0ade172dd4bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:11:12.437695 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d375f-334b-47f5-8681-0ade172dd4bc service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:12.437970 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d375f-334b-47f5-8681-0ade172dd4bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:12.438262 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d375f-334b-47f5-8681-0ade172dd4bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:12.439859 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-9a9d375f-334b-47f5-8681-0ade172dd4bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:12.554399 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9d375f-334b-47f5-8681-0ade172dd4bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:11:12.555843 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9d375f-334b-47f5-8681-0ade172dd4bc service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:11:12.574427 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 661/1340] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:11:12 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:11:16.394661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf65784-7dbe-43a4-ad75-5a0a11487ca2 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:16.395143 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf65784-7dbe-43a4-ad75-5a0a11487ca2 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:16.395331 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf65784-7dbe-43a4-ad75-5a0a11487ca2 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:16.397467 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 680/1341] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:11:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:11:16.403916 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e8c48-d44c-4f58-b6bf-3af3191e496d None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:16.403916 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e8c48-d44c-4f58-b6bf-3af3191e496d None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:16.404365 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e8c48-d44c-4f58-b6bf-3af3191e496d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:16.409824 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:11:16.478350 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-856e8c48-d44c-4f58-b6bf-3af3191e496d None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:11:16.479596 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-856e8c48-d44c-4f58-b6bf-3af3191e496d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:16.575958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 662/1342] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:11:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:11:16.853623 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf65784-7dbe-43a4-ad75-5a0a11487ca2 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:11:16.855247 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fbf65784-7dbe-43a4-ad75-5a0a11487ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:16.904127 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:11:16.961485 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf65784-7dbe-43a4-ad75-5a0a11487ca2 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:11:17.040286 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1d96cf-580d-4e60-9ee5-0d82f36eb640 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:11:17.040891 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d96cf-580d-4e60-9ee5-0d82f36eb640 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:17.041166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d96cf-580d-4e60-9ee5-0d82f36eb640 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:17.041431 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d96cf-580d-4e60-9ee5-0d82f36eb640 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:17.043049 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ce1d96cf-580d-4e60-9ee5-0d82f36eb640 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:17.138442 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:11:17.156504 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1d96cf-580d-4e60-9ee5-0d82f36eb640 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:11:17.157958 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1d96cf-580d-4e60-9ee5-0d82f36eb640 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:11:17.176698 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 681/1343] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:11:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 05:11:20.058624 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503ab9a3-3624-4f10-b4ef-5746b62f2fca None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:20.059199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503ab9a3-3624-4f10-b4ef-5746b62f2fca None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:20.059199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503ab9a3-3624-4f10-b4ef-5746b62f2fca None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:20.060687 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 663/1344] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:11:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:11:20.066350 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e1df5-a8d4-456d-b52b-9cc75dc31280 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:20.066818 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e1df5-a8d4-456d-b52b-9cc75dc31280 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:20.066924 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e1df5-a8d4-456d-b52b-9cc75dc31280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:20.138454 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-522e1df5-a8d4-456d-b52b-9cc75dc31280 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:11:20.139728 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-522e1df5-a8d4-456d-b52b-9cc75dc31280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:20.239838 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 682/1345] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:11:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:11:20.517407 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503ab9a3-3624-4f10-b4ef-5746b62f2fca None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:11:20.518937 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-503ab9a3-3624-4f10-b4ef-5746b62f2fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:20.619608 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:11:20.640568 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503ab9a3-3624-4f10-b4ef-5746b62f2fca None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:11:20.725174 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57179333-b715-4fc8-8bf3-565a439db218 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:11:20.725774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57179333-b715-4fc8-8bf3-565a439db218 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:20.726053 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57179333-b715-4fc8-8bf3-565a439db218 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:20.726360 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57179333-b715-4fc8-8bf3-565a439db218 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:20.727909 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-57179333-b715-4fc8-8bf3-565a439db218 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:20.838489 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57179333-b715-4fc8-8bf3-565a439db218 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:11:20.839863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57179333-b715-4fc8-8bf3-565a439db218 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:11:20.859294 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 664/1346] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:11:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:11:24.821031 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44ec53-d424-47ce-8929-acc0f74cd013 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:24.821604 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44ec53-d424-47ce-8929-acc0f74cd013 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:24.821604 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44ec53-d424-47ce-8929-acc0f74cd013 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:24.822955 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 683/1347] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:11:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:11:24.828493 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73705e11-0ad8-44e1-a70a-15ee30a7bd61 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:24.829003 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73705e11-0ad8-44e1-a70a-15ee30a7bd61 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:24.829003 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73705e11-0ad8-44e1-a70a-15ee30a7bd61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:24.845153 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:11:24.900469 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-73705e11-0ad8-44e1-a70a-15ee30a7bd61 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:11:24.901685 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-73705e11-0ad8-44e1-a70a-15ee30a7bd61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:25.010213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 665/1348] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:11:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:11:25.292109 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf44ec53-d424-47ce-8929-acc0f74cd013 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:11:25.293622 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-cf44ec53-d424-47ce-8929-acc0f74cd013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:25.353348 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:11:25.401544 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf44ec53-d424-47ce-8929-acc0f74cd013 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:11:25.484127 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffdbd21-e351-4f57-8819-9e7b1539d01f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:11:25.484767 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffdbd21-e351-4f57-8819-9e7b1539d01f service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:25.485038 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffdbd21-e351-4f57-8819-9e7b1539d01f service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:25.485303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffdbd21-e351-4f57-8819-9e7b1539d01f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:25.486920 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-dffdbd21-e351-4f57-8819-9e7b1539d01f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:25.603448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:11:25.606786 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffdbd21-e351-4f57-8819-9e7b1539d01f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:11:25.608576 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffdbd21-e351-4f57-8819-9e7b1539d01f service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:11:25.627762 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 684/1349] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:11:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 05:11:30.526487 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217d812-37cd-41b7-8f80-8c0cf619c548 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:30.526995 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217d812-37cd-41b7-8f80-8c0cf619c548 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:30.527095 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217d812-37cd-41b7-8f80-8c0cf619c548 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:30.528852 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 666/1350] 10.222.0.36 () {54 vars in 921 bytes} [Wed Aug 31 05:11:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:11:30.534892 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9804d0-5db7-4506-b7f8-f053ee2a524f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:30.535335 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9804d0-5db7-4506-b7f8-f053ee2a524f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:30.536324 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9804d0-5db7-4506-b7f8-f053ee2a524f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:30.636939 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-0f9804d0-5db7-4506-b7f8-f053ee2a524f None None] MFA Rules not processed for user `bd4387035099416789009f0ecb8ad282`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:11:30.638819 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0f9804d0-5db7-4506-b7f8-f053ee2a524f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:30.797078 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 685/1351] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Aug 31 05:11:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:11:34.829141 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b04948-1ea6-49dd-9936-a09c8dbd9fe2 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:34.829742 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b04948-1ea6-49dd-9936-a09c8dbd9fe2 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:34.829742 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b04948-1ea6-49dd-9936-a09c8dbd9fe2 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:34.831207 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 667/1352] 10.222.0.36 () {54 vars in 921 bytes} [Wed Aug 31 05:11:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:11:34.836193 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5bf06d-2409-44bf-8a16-9c1416c9432b None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:34.836667 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5bf06d-2409-44bf-8a16-9c1416c9432b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:34.836961 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5bf06d-2409-44bf-8a16-9c1416c9432b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:34.895911 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:11:34.921359 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-5e5bf06d-2409-44bf-8a16-9c1416c9432b None None] MFA Rules not processed for user `bd4387035099416789009f0ecb8ad282`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:11:34.922909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5e5bf06d-2409-44bf-8a16-9c1416c9432b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:35.008220 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=59 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:11:35.033532 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 686/1353] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Aug 31 05:11:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:11:36.400960 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1958da-3d7a-4d5c-b922-b5431979c487 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:36.401485 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1958da-3d7a-4d5c-b922-b5431979c487 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:36.401485 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1958da-3d7a-4d5c-b922-b5431979c487 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:36.402965 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 668/1354] 10.222.0.36 () {54 vars in 921 bytes} [Wed Aug 31 05:11:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:11:36.408270 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec5941-13f8-4928-8c85-ed5191ebfd08 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:36.408684 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec5941-13f8-4928-8c85-ed5191ebfd08 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:36.408846 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec5941-13f8-4928-8c85-ed5191ebfd08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:36.489336 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-78ec5941-13f8-4928-8c85-ed5191ebfd08 None None] MFA Rules not processed for user `bd4387035099416789009f0ecb8ad282`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:11:36.490571 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-78ec5941-13f8-4928-8c85-ed5191ebfd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:36.533721 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:11:36.600067 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 687/1355] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Aug 31 05:11:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:11:39.865461 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d528091a-441d-43de-a00c-8ccef2d282d1 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:39.866086 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d528091a-441d-43de-a00c-8ccef2d282d1 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:39.866086 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d528091a-441d-43de-a00c-8ccef2d282d1 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:39.867629 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 669/1356] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:11:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:11:39.872620 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea16bdd-e09a-47df-bf23-54bb8ea5c0c0 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:39.873643 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea16bdd-e09a-47df-bf23-54bb8ea5c0c0 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:39.873643 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea16bdd-e09a-47df-bf23-54bb8ea5c0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:39.951434 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-8ea16bdd-e09a-47df-bf23-54bb8ea5c0c0 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:11:39.955780 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8ea16bdd-e09a-47df-bf23-54bb8ea5c0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:40.074829 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 688/1357] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:11:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:11:40.371520 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0d81d-93b2-4f14-bf59-ba1c4dbe0e26 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:40.371819 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0d81d-93b2-4f14-bf59-ba1c4dbe0e26 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:40.372075 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0d81d-93b2-4f14-bf59-ba1c4dbe0e26 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:40.373478 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 670/1358] 10.222.0.36 () {54 vars in 962 bytes} [Wed Aug 31 05:11:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:11:40.382783 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7200609c-a993-4c5d-8966-1b4c5ca96dff None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:40.383189 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7200609c-a993-4c5d-8966-1b4c5ca96dff None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:40.383562 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7200609c-a993-4c5d-8966-1b4c5ca96dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:40.402313 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:11:40.467558 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-7200609c-a993-4c5d-8966-1b4c5ca96dff None None] MFA Rules not processed for user `f27665b1574a4db7b7a9771f7d775826`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:11:40.468865 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7200609c-a993-4c5d-8966-1b4c5ca96dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:40.593804 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 689/1359] 10.222.0.36 () {58 vars in 1059 bytes} [Wed Aug 31 05:11:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:11:40.602027 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d0d81d-93b2-4f14-bf59-ba1c4dbe0e26 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:11:40.603483 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-53d0d81d-93b2-4f14-bf59-ba1c4dbe0e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:40.760804 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d0d81d-93b2-4f14-bf59-ba1c4dbe0e26 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:11:40.841101 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:11:40.883855 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c299a97b-c939-4d7e-a9f9-be850ad8fe95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:11:40.884491 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299a97b-c939-4d7e-a9f9-be850ad8fe95 service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:40.884717 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299a97b-c939-4d7e-a9f9-be850ad8fe95 service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:40.884895 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299a97b-c939-4d7e-a9f9-be850ad8fe95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:40.886530 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c299a97b-c939-4d7e-a9f9-be850ad8fe95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:40.977054 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c299a97b-c939-4d7e-a9f9-be850ad8fe95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:11:40.979471 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c299a97b-c939-4d7e-a9f9-be850ad8fe95 service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:11:40.998533 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 671/1360] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:11:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:11:43.112681 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d556dd-f787-4594-b270-20664f532f24 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:43.113222 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d556dd-f787-4594-b270-20664f532f24 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:43.113467 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d556dd-f787-4594-b270-20664f532f24 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:43.115003 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 690/1361] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:11:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:11:43.120457 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9f034-8003-4377-ba30-2314d153f878 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:43.120893 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9f034-8003-4377-ba30-2314d153f878 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:43.120974 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9f034-8003-4377-ba30-2314d153f878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:43.194378 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-2ce9f034-8003-4377-ba30-2314d153f878 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:11:43.195599 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-2ce9f034-8003-4377-ba30-2314d153f878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:43.237123 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:11:43.308382 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 672/1362] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:11:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:11:43.603853 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d556dd-f787-4594-b270-20664f532f24 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:11:43.605447 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-08d556dd-f787-4594-b270-20664f532f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:43.687632 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:11:43.780088 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d556dd-f787-4594-b270-20664f532f24 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:11:43.916403 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d4325-84ed-4327-88ce-eab4317de269 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:11:43.917078 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d4325-84ed-4327-88ce-eab4317de269 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:43.917421 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d4325-84ed-4327-88ce-eab4317de269 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:43.917754 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d4325-84ed-4327-88ce-eab4317de269 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:43.919635 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-300d4325-84ed-4327-88ce-eab4317de269 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:43.942727 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:11:44.010954 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300d4325-84ed-4327-88ce-eab4317de269 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:11:44.012631 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300d4325-84ed-4327-88ce-eab4317de269 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:11:44.031510 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 691/1363] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:11:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:11:46.226587 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9eec5-7522-4ab8-bc77-029517812adb None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:46.227158 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9eec5-7522-4ab8-bc77-029517812adb None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:46.227158 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9eec5-7522-4ab8-bc77-029517812adb None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:46.228708 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 673/1364] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:11:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:11:46.234840 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf2218-8699-42f7-88c0-1a99fbeaf3cf None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:46.235653 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf2218-8699-42f7-88c0-1a99fbeaf3cf None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:46.235902 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf2218-8699-42f7-88c0-1a99fbeaf3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:46.320845 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-35bf2218-8699-42f7-88c0-1a99fbeaf3cf None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:11:46.322223 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-35bf2218-8699-42f7-88c0-1a99fbeaf3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:46.425110 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:11:46.433886 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 692/1365] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:11:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:11:46.729013 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd005a1-04bd-4bd8-be00-9a2a2d6c5e65 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:46.729982 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd005a1-04bd-4bd8-be00-9a2a2d6c5e65 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:46.730521 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd005a1-04bd-4bd8-be00-9a2a2d6c5e65 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:46.732697 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 674/1366] 10.222.0.36 () {54 vars in 962 bytes} [Wed Aug 31 05:11:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:11:46.738131 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc3686-e3f3-4fda-97df-ad0042984cf0 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:46.738401 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc3686-e3f3-4fda-97df-ad0042984cf0 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:46.738713 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc3686-e3f3-4fda-97df-ad0042984cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:46.818761 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-74dc3686-e3f3-4fda-97df-ad0042984cf0 None None] MFA Rules not processed for user `f27665b1574a4db7b7a9771f7d775826`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:11:46.820157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-74dc3686-e3f3-4fda-97df-ad0042984cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:46.945763 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 693/1367] 10.222.0.36 () {58 vars in 1059 bytes} [Wed Aug 31 05:11:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:11:46.955139 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd005a1-04bd-4bd8-be00-9a2a2d6c5e65 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:11:46.956791 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-bdd005a1-04bd-4bd8-be00-9a2a2d6c5e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:47.117668 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd005a1-04bd-4bd8-be00-9a2a2d6c5e65 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:11:47.138632 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:11:47.233863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013c16f4-41d3-4412-9fd0-fee3d8071e75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:11:47.234560 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c16f4-41d3-4412-9fd0-fee3d8071e75 service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:47.234666 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c16f4-41d3-4412-9fd0-fee3d8071e75 service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:47.234953 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c16f4-41d3-4412-9fd0-fee3d8071e75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:47.236496 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-013c16f4-41d3-4412-9fd0-fee3d8071e75 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:47.334874 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013c16f4-41d3-4412-9fd0-fee3d8071e75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:11:47.336479 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013c16f4-41d3-4412-9fd0-fee3d8071e75 service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:11:47.356113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 675/1368] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:11:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:11:47.896313 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26686023-2af2-4a7f-bff6-1d2d3c091c23 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:47.896863 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26686023-2af2-4a7f-bff6-1d2d3c091c23 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:47.896863 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26686023-2af2-4a7f-bff6-1d2d3c091c23 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:47.898162 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 694/1369] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:11:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:11:47.903573 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a13aea-5480-4169-9fa3-08530a526ef2 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:47.903851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a13aea-5480-4169-9fa3-08530a526ef2 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:47.904116 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a13aea-5480-4169-9fa3-08530a526ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:47.937348 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:11:47.977680 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-92a13aea-5480-4169-9fa3-08530a526ef2 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:11:47.978909 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-92a13aea-5480-4169-9fa3-08530a526ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:48.090074 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 676/1370] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:11:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:11:48.672523 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d69ed-443a-4f5c-b539-3eda0cf06bd5 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:48.672523 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d69ed-443a-4f5c-b539-3eda0cf06bd5 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:48.672523 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d69ed-443a-4f5c-b539-3eda0cf06bd5 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:48.673827 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 695/1371] 10.222.0.36 () {54 vars in 964 bytes} [Wed Aug 31 05:11:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:11:48.678749 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366338f-2b4e-4e02-b258-1ecba06028ba None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:48.679309 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366338f-2b4e-4e02-b258-1ecba06028ba None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:48.679458 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366338f-2b4e-4e02-b258-1ecba06028ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:48.761434 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-3366338f-2b4e-4e02-b258-1ecba06028ba None None] MFA Rules not processed for user `a8300075371341a6b9cd4dcf8ec7fa8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:11:48.762843 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3366338f-2b4e-4e02-b258-1ecba06028ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:48.810896 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:11:48.842255 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 677/1372] 10.222.0.36 () {58 vars in 1061 bytes} [Wed Aug 31 05:11:48 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:11:48.850485 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d69ed-443a-4f5c-b539-3eda0cf06bd5 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:11:48.852084 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-910d69ed-443a-4f5c-b539-3eda0cf06bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:48.905974 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:11:48.962295 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d69ed-443a-4f5c-b539-3eda0cf06bd5 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:11:49.043858 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb1470b-70c9-49c4-9f5e-525ac1e76ee2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:11:49.044585 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb1470b-70c9-49c4-9f5e-525ac1e76ee2 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:49.044905 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb1470b-70c9-49c4-9f5e-525ac1e76ee2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:49.045121 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb1470b-70c9-49c4-9f5e-525ac1e76ee2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:49.046866 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6bb1470b-70c9-49c4-9f5e-525ac1e76ee2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:49.154435 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:11:49.171515 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb1470b-70c9-49c4-9f5e-525ac1e76ee2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:11:49.173023 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb1470b-70c9-49c4-9f5e-525ac1e76ee2 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:11:49.199310 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 696/1373] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:11:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:11:49.334762 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b04754-2895-4e56-9ae7-cfd891ea15b8 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:49.334958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b04754-2895-4e56-9ae7-cfd891ea15b8 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:49.335170 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b04754-2895-4e56-9ae7-cfd891ea15b8 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:49.336526 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 678/1374] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:11:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:11:49.342171 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d189b28d-3192-4c40-a11d-1b05c1a2dbb8 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:49.342453 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d189b28d-3192-4c40-a11d-1b05c1a2dbb8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:49.342720 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d189b28d-3192-4c40-a11d-1b05c1a2dbb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:49.416057 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d189b28d-3192-4c40-a11d-1b05c1a2dbb8 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:11:49.417213 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d189b28d-3192-4c40-a11d-1b05c1a2dbb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:49.519985 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 697/1375] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:11:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:11:49.800685 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b04754-2895-4e56-9ae7-cfd891ea15b8 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:11:49.802353 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-d2b04754-2895-4e56-9ae7-cfd891ea15b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:49.958163 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b04754-2895-4e56-9ae7-cfd891ea15b8 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:11:50.010570 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:11:50.074842 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf87161b-9f9a-4743-976d-9979696c125d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:11:50.075398 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf87161b-9f9a-4743-976d-9979696c125d service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:50.075674 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf87161b-9f9a-4743-976d-9979696c125d service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:50.075938 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf87161b-9f9a-4743-976d-9979696c125d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:50.077521 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-cf87161b-9f9a-4743-976d-9979696c125d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:50.166586 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf87161b-9f9a-4743-976d-9979696c125d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:11:50.168202 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf87161b-9f9a-4743-976d-9979696c125d service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:11:50.186992 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 679/1376] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:11:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:11:52.112842 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a5044-8437-4472-a1eb-75f4d6bf58e6 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:52.113353 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a5044-8437-4472-a1eb-75f4d6bf58e6 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:52.113450 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a5044-8437-4472-a1eb-75f4d6bf58e6 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:52.114778 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 698/1377] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:11:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:11:52.120717 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4c110-954e-4c0e-a6f4-cd04b978c596 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:52.121451 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4c110-954e-4c0e-a6f4-cd04b978c596 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:52.121613 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4c110-954e-4c0e-a6f4-cd04b978c596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:52.208956 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-faf4c110-954e-4c0e-a6f4-cd04b978c596 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:11:52.210567 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-faf4c110-954e-4c0e-a6f4-cd04b978c596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:52.217004 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:11:52.308763 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 680/1378] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:11:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:11:52.616503 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58a5044-8437-4472-a1eb-75f4d6bf58e6 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:11:52.617947 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a58a5044-8437-4472-a1eb-75f4d6bf58e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:52.714686 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:11:52.811410 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58a5044-8437-4472-a1eb-75f4d6bf58e6 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:11:52.930631 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06abdfb-a905-4933-aac4-ed6dcd9c2ff3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:11:52.931154 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06abdfb-a905-4933-aac4-ed6dcd9c2ff3 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:52.931428 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06abdfb-a905-4933-aac4-ed6dcd9c2ff3 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:52.931689 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06abdfb-a905-4933-aac4-ed6dcd9c2ff3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:52.933343 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a06abdfb-a905-4933-aac4-ed6dcd9c2ff3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:52.953978 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:11:53.027323 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06abdfb-a905-4933-aac4-ed6dcd9c2ff3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:11:53.028934 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06abdfb-a905-4933-aac4-ed6dcd9c2ff3 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:11:53.047823 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 699/1379] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:11:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:11:54.967693 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5e9d9-6170-47dc-a36d-248904a128ef None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:54.968257 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5e9d9-6170-47dc-a36d-248904a128ef None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:54.968257 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5e9d9-6170-47dc-a36d-248904a128ef None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:54.969776 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 681/1380] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:11:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:11:54.974853 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b21b8-049f-4c5f-8b0c-31c3a0d7155f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:54.975332 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b21b8-049f-4c5f-8b0c-31c3a0d7155f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:54.975426 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b21b8-049f-4c5f-8b0c-31c3a0d7155f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:55.045851 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-bc6b21b8-049f-4c5f-8b0c-31c3a0d7155f None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:11:55.047090 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-bc6b21b8-049f-4c5f-8b0c-31c3a0d7155f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:55.142293 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:11:55.154050 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 700/1381] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:11:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:11:55.436739 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5e9d9-6170-47dc-a36d-248904a128ef None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:11:55.438289 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-89a5e9d9-6170-47dc-a36d-248904a128ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:55.576613 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:11:55.588182 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5e9d9-6170-47dc-a36d-248904a128ef None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:11:55.714452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9f423-db23-492b-ad44-80f9a9f2d9df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:11:55.714978 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9f423-db23-492b-ad44-80f9a9f2d9df service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:55.715252 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9f423-db23-492b-ad44-80f9a9f2d9df service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:55.715522 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9f423-db23-492b-ad44-80f9a9f2d9df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:55.717106 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-f5c9f423-db23-492b-ad44-80f9a9f2d9df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:55.809749 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c9f423-db23-492b-ad44-80f9a9f2d9df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:11:55.811371 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c9f423-db23-492b-ad44-80f9a9f2d9df service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:11:55.822245 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:11:55.831237 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 682/1382] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:11:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:11:58.072416 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6803f3d-b334-4f9c-a04a-d9615858a571 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:58.072992 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6803f3d-b334-4f9c-a04a-d9615858a571 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:58.073170 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6803f3d-b334-4f9c-a04a-d9615858a571 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:58.075076 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 701/1383] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:11:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 05:11:58.080997 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67a407-26c2-43c7-a7b3-27d5e17ce5e6 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:58.081450 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67a407-26c2-43c7-a7b3-27d5e17ce5e6 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:58.081591 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67a407-26c2-43c7-a7b3-27d5e17ce5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:58.161165 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-3b67a407-26c2-43c7-a7b3-27d5e17ce5e6 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:11:58.162763 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3b67a407-26c2-43c7-a7b3-27d5e17ce5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:58.276682 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 683/1384] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:11:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:11:58.563191 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6803f3d-b334-4f9c-a04a-d9615858a571 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:11:58.565242 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e6803f3d-b334-4f9c-a04a-d9615858a571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:58.728274 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6803f3d-b334-4f9c-a04a-d9615858a571 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:11:58.805745 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:11:58.854627 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ff313e-01da-4b75-8db3-cc1ff97e6833 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:11:58.855790 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff313e-01da-4b75-8db3-cc1ff97e6833 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:11:58.856439 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff313e-01da-4b75-8db3-cc1ff97e6833 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:11:58.856694 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff313e-01da-4b75-8db3-cc1ff97e6833 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:11:58.858683 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-25ff313e-01da-4b75-8db3-cc1ff97e6833 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:11:58.948642 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ff313e-01da-4b75-8db3-cc1ff97e6833 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:11:58.950316 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ff313e-01da-4b75-8db3-cc1ff97e6833 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:11:58.970346 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 702/1385] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:11:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 05:12:01.120428 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac244215-d010-44be-b87e-4f1587d6f2b5 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:01.121023 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac244215-d010-44be-b87e-4f1587d6f2b5 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:01.121166 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac244215-d010-44be-b87e-4f1587d6f2b5 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:01.123038 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 684/1386] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:12:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:12:01.129343 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1c260-0db3-4f94-8a4a-26f520529ea8 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:01.129811 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1c260-0db3-4f94-8a4a-26f520529ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:01.129928 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1c260-0db3-4f94-8a4a-26f520529ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:01.200468 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-67b1c260-0db3-4f94-8a4a-26f520529ea8 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:12:01.201679 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-67b1c260-0db3-4f94-8a4a-26f520529ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:01.240200 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:01.304695 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 703/1387] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:12:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:12:01.666739 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac244215-d010-44be-b87e-4f1587d6f2b5 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:12:01.668429 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-ac244215-d010-44be-b87e-4f1587d6f2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:01.746422 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:01.816323 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac244215-d010-44be-b87e-4f1587d6f2b5 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:12:01.959158 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f4a7d-371a-4de9-8930-cb8911e1bf65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:12:01.959158 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f4a7d-371a-4de9-8930-cb8911e1bf65 service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:01.959158 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f4a7d-371a-4de9-8930-cb8911e1bf65 service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:01.959158 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f4a7d-371a-4de9-8930-cb8911e1bf65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:01.961094 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-4b6f4a7d-371a-4de9-8930-cb8911e1bf65 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:01.999948 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:02.055069 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6f4a7d-371a-4de9-8930-cb8911e1bf65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:12:02.056701 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6f4a7d-371a-4de9-8930-cb8911e1bf65 service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:12:02.075238 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 685/1388] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:12:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:12:04.002928 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a50546-4548-4b09-8270-ae0fd68ee6e6 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:04.003554 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a50546-4548-4b09-8270-ae0fd68ee6e6 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:04.003554 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a50546-4548-4b09-8270-ae0fd68ee6e6 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:04.004837 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 704/1389] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:12:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:12:04.010054 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22e4aa-ede1-46cf-9cd3-2c027a930671 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:04.010498 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13df4b-c1ac-4d49-ab16-2c89655c1eb4 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:04.010498 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13df4b-c1ac-4d49-ab16-2c89655c1eb4 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:04.010681 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22e4aa-ede1-46cf-9cd3-2c027a930671 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:04.010765 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13df4b-c1ac-4d49-ab16-2c89655c1eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:04.010853 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22e4aa-ede1-46cf-9cd3-2c027a930671 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:04.012128 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 686/1390] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:12:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:12:04.016833 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c323c-abbe-473f-901b-554f3e200341 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:04.017116 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c323c-abbe-473f-901b-554f3e200341 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:04.017355 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c323c-abbe-473f-901b-554f3e200341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:04.080348 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-5e13df4b-c1ac-4d49-ab16-2c89655c1eb4 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:12:04.081559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5e13df4b-c1ac-4d49-ab16-2c89655c1eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:04.088389 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-817c323c-abbe-473f-901b-554f3e200341 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:12:04.089536 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-817c323c-abbe-473f-901b-554f3e200341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:04.179064 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 705/1391] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:12:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:12:04.191330 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 687/1392] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:12:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:12:04.464037 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13df4b-c1ac-4d49-ab16-2c89655c1eb4 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:12:04.465319 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5e13df4b-c1ac-4d49-ab16-2c89655c1eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:04.479267 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:04.483651 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd2778-1e6d-443e-888b-8fe9e9463daf None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:04.483933 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd2778-1e6d-443e-888b-8fe9e9463daf None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:04.484204 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd2778-1e6d-443e-888b-8fe9e9463daf None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:04.485561 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 688/1393] 10.222.0.36 () {54 vars in 967 bytes} [Wed Aug 31 05:12:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:12:04.490648 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd439a7c-4dcf-4646-8564-fc81d0c1ba0c None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:04.490930 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd439a7c-4dcf-4646-8564-fc81d0c1ba0c None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:04.491200 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd439a7c-4dcf-4646-8564-fc81d0c1ba0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:04.508052 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:04.572912 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-dd439a7c-4dcf-4646-8564-fc81d0c1ba0c None None] MFA Rules not processed for user `c08e6024b0c8420aa274653c2a916edd`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:12:04.574209 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-dd439a7c-4dcf-4646-8564-fc81d0c1ba0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:04.585393 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13df4b-c1ac-4d49-ab16-2c89655c1eb4 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:12:04.657218 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 689/1394] 10.222.0.36 () {58 vars in 1064 bytes} [Wed Aug 31 05:12:04 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:12:04.664636 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd439a7c-4dcf-4646-8564-fc81d0c1ba0c None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:12:04.672224 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd439a7c-4dcf-4646-8564-fc81d0c1ba0c None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:12:04.700453 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7defe8fe-b669-4718-991a-02336ecd8356 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:12:04.701341 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7defe8fe-b669-4718-991a-02336ecd8356 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:04.701460 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7defe8fe-b669-4718-991a-02336ecd8356 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:04.701624 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7defe8fe-b669-4718-991a-02336ecd8356 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:04.703256 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7defe8fe-b669-4718-991a-02336ecd8356 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:04.712163 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:04.748055 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa47cd1-5824-4049-92e2-1e30cb4ba629 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08e6024b0c8420aa274653c2a916edd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:12:04.748662 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa47cd1-5824-4049-92e2-1e30cb4ba629 service glance] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:04.748936 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa47cd1-5824-4049-92e2-1e30cb4ba629 service glance] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:04.749262 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa47cd1-5824-4049-92e2-1e30cb4ba629 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:04.750910 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-4aa47cd1-5824-4049-92e2-1e30cb4ba629 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:04.780409 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:04.792241 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7defe8fe-b669-4718-991a-02336ecd8356 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:12:04.793904 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7defe8fe-b669-4718-991a-02336ecd8356 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:12:04.811752 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 706/1395] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:12:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:12:04.864049 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa47cd1-5824-4049-92e2-1e30cb4ba629 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:12:04.865584 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa47cd1-5824-4049-92e2-1e30cb4ba629 service glance] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:12:04.884516 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 690/1396] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:12:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 05:12:05.621133 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b2aba4-d21b-4183-a9c5-2f8e830f5842 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:05.621600 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b2aba4-d21b-4183-a9c5-2f8e830f5842 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:05.621685 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b2aba4-d21b-4183-a9c5-2f8e830f5842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:05.691130 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-70b2aba4-d21b-4183-a9c5-2f8e830f5842 None None] MFA Rules not processed for user `09bf6db88ce8494a937211a814c9ff84`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:12:05.692637 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-70b2aba4-d21b-4183-a9c5-2f8e830f5842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:05.779064 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:05.782800 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 707/1397] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Aug 31 05:12:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:12:05.823433 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a87e6-2269-4a35-9759-83fc8a24b603 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:05.824001 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a87e6-2269-4a35-9759-83fc8a24b603 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:05.824001 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a87e6-2269-4a35-9759-83fc8a24b603 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:05.825612 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 691/1398] 10.222.0.36 () {54 vars in 943 bytes} [Wed Aug 31 05:12:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:12:05.831681 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b115b-faa5-4170-93cf-42a6e6c6edbb None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:05.831962 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b115b-faa5-4170-93cf-42a6e6c6edbb None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:05.832233 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b115b-faa5-4170-93cf-42a6e6c6edbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:05.909911 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ee2b115b-faa5-4170-93cf-42a6e6c6edbb None None] MFA Rules not processed for user `d9b82080a73d4c4a8ec08b0e67d6d11c`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:12:05.911114 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ee2b115b-faa5-4170-93cf-42a6e6c6edbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:06.022183 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 708/1399] 10.222.0.36 () {58 vars in 1040 bytes} [Wed Aug 31 05:12:05 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:12:06.030314 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a87e6-2269-4a35-9759-83fc8a24b603 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:12:06.031847 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-443a87e6-2269-4a35-9759-83fc8a24b603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:06.167057 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:06.169843 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a87e6-2269-4a35-9759-83fc8a24b603 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:12:06.282485 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc03c231-479a-4a16-bc32-403e978f59c1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9b82080a73d4c4a8ec08b0e67d6d11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:12:06.283094 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03c231-479a-4a16-bc32-403e978f59c1 service swift] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:06.283384 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03c231-479a-4a16-bc32-403e978f59c1 service swift] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:06.283642 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03c231-479a-4a16-bc32-403e978f59c1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:06.285265 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-fc03c231-479a-4a16-bc32-403e978f59c1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:06.368743 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc03c231-479a-4a16-bc32-403e978f59c1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:12:06.370443 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc03c231-479a-4a16-bc32-403e978f59c1 service swift] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:12:06.379104 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 692/1400] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Aug 31 05:12:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 05:12:06.736986 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c50ed-f50d-4021-b4a7-0b8e5e2017c3 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:06.737514 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c50ed-f50d-4021-b4a7-0b8e5e2017c3 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:06.737514 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c50ed-f50d-4021-b4a7-0b8e5e2017c3 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:06.739016 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 709/1401] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:12:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:12:06.744195 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e5b2f-a175-402b-9ffe-82d105fb384d None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:06.744339 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e5b2f-a175-402b-9ffe-82d105fb384d None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:06.744580 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e5b2f-a175-402b-9ffe-82d105fb384d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:06.787250 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:06.824633 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-089e5b2f-a175-402b-9ffe-82d105fb384d None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:12:06.825858 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-089e5b2f-a175-402b-9ffe-82d105fb384d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:06.925273 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 693/1402] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:12:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:12:07.214934 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16c50ed-f50d-4021-b4a7-0b8e5e2017c3 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:12:07.216461 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a16c50ed-f50d-4021-b4a7-0b8e5e2017c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:07.290950 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:07.388678 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16c50ed-f50d-4021-b4a7-0b8e5e2017c3 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:12:07.515450 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458d3078-717c-41db-a235-9c36c7b239f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:12:07.516078 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d3078-717c-41db-a235-9c36c7b239f3 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:07.516356 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d3078-717c-41db-a235-9c36c7b239f3 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:07.516623 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d3078-717c-41db-a235-9c36c7b239f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:07.519634 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-458d3078-717c-41db-a235-9c36c7b239f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:07.541493 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:07.620730 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458d3078-717c-41db-a235-9c36c7b239f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:12:07.622387 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458d3078-717c-41db-a235-9c36c7b239f3 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:12:07.641357 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 710/1403] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:12:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:12:09.596168 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8333316-596a-48aa-b041-a28969a5601c None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:09.596723 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8333316-596a-48aa-b041-a28969a5601c None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:09.596723 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8333316-596a-48aa-b041-a28969a5601c None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:09.598427 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 694/1404] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:12:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:12:09.603510 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc869d-5f70-4506-924a-05940c04908f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:09.603930 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc869d-5f70-4506-924a-05940c04908f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:09.604036 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc869d-5f70-4506-924a-05940c04908f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:09.688713 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a8bc869d-5f70-4506-924a-05940c04908f None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:12:09.689941 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a8bc869d-5f70-4506-924a-05940c04908f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:09.795341 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:09.803475 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 711/1405] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:12:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:12:10.172205 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8333316-596a-48aa-b041-a28969a5601c None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:12:10.173864 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c8333316-596a-48aa-b041-a28969a5601c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:10.300019 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:10.358285 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8333316-596a-48aa-b041-a28969a5601c None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:12:10.525893 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab0ccfd-6852-41eb-ab48-855726a20b50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:12:10.526916 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0ccfd-6852-41eb-ab48-855726a20b50 service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:10.527380 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0ccfd-6852-41eb-ab48-855726a20b50 service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:10.527969 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0ccfd-6852-41eb-ab48-855726a20b50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:10.530243 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-2ab0ccfd-6852-41eb-ab48-855726a20b50 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:10.615913 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:10.639301 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab0ccfd-6852-41eb-ab48-855726a20b50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:12:10.641531 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab0ccfd-6852-41eb-ab48-855726a20b50 service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:12:10.661939 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 695/1406] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:12:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:12:12.612024 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81046d24-a818-4b7e-adb4-3896b81e8d8f None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:12.612608 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81046d24-a818-4b7e-adb4-3896b81e8d8f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:12.612608 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81046d24-a818-4b7e-adb4-3896b81e8d8f None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:12.616943 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 712/1407] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:12:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:12:12.619209 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be7e95-14d2-4fe4-9aa2-649070944102 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:12.619608 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be7e95-14d2-4fe4-9aa2-649070944102 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:12.619967 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be7e95-14d2-4fe4-9aa2-649070944102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:12.699765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-94be7e95-14d2-4fe4-9aa2-649070944102 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:12:12.700969 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-94be7e95-14d2-4fe4-9aa2-649070944102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:12.810062 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 696/1408] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:12:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:12:13.097441 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81046d24-a818-4b7e-adb4-3896b81e8d8f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:12:13.099009 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-81046d24-a818-4b7e-adb4-3896b81e8d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:13.269543 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81046d24-a818-4b7e-adb4-3896b81e8d8f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:12:13.343950 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:13.400670 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7a8a66-6ceb-403e-8264-0e23f0fc471e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:12:13.401697 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a8a66-6ceb-403e-8264-0e23f0fc471e service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:13.401975 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a8a66-6ceb-403e-8264-0e23f0fc471e service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:13.402289 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a8a66-6ceb-403e-8264-0e23f0fc471e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:13.404020 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-df7a8a66-6ceb-403e-8264-0e23f0fc471e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:13.504977 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7a8a66-6ceb-403e-8264-0e23f0fc471e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:12:13.506756 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7a8a66-6ceb-403e-8264-0e23f0fc471e service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:12:13.529102 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 713/1409] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:12:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:12:22.807539 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=79 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:23.523042 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=64 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:28.161961 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388849c-cbf9-457d-a846-f65c14c0a390 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:28.162614 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388849c-cbf9-457d-a846-f65c14c0a390 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:28.162614 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388849c-cbf9-457d-a846-f65c14c0a390 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:28.163982 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 697/1410] 10.222.0.36 () {54 vars in 918 bytes} [Wed Aug 31 05:12:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:12:28.170907 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110cf9d-499f-4bf9-b123-766850f436ef None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:28.171888 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110cf9d-499f-4bf9-b123-766850f436ef None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:28.171888 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110cf9d-499f-4bf9-b123-766850f436ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:28.273304 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-3110cf9d-499f-4bf9-b123-766850f436ef None None] MFA Rules not processed for user `bd4387035099416789009f0ecb8ad282`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:12:28.274573 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3110cf9d-499f-4bf9-b123-766850f436ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:28.387145 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 714/1411] 10.222.0.36 () {58 vars in 1015 bytes} [Wed Aug 31 05:12:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:12:28.401490 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face2e06-2ebf-4673-9fda-4f007297aeff None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:28.401826 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face2e06-2ebf-4673-9fda-4f007297aeff None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:28.402152 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face2e06-2ebf-4673-9fda-4f007297aeff None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:28.403528 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 698/1412] 10.222.0.36 () {54 vars in 918 bytes} [Wed Aug 31 05:12:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:12:28.408282 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7764c8-d25d-417a-be87-14606b1a51ae None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:28.408568 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7764c8-d25d-417a-be87-14606b1a51ae None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:28.408838 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7764c8-d25d-417a-be87-14606b1a51ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:28.489407 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:28.492890 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-af7764c8-d25d-417a-be87-14606b1a51ae None None] MFA Rules not processed for user `f27665b1574a4db7b7a9771f7d775826`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:12:28.494344 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-af7764c8-d25d-417a-be87-14606b1a51ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:28.626325 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 715/1413] 10.222.0.36 () {58 vars in 1015 bytes} [Wed Aug 31 05:12:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:12:31.017996 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01111d30-3f60-43e4-90db-e559a82a589a None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:31.018886 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01111d30-3f60-43e4-90db-e559a82a589a None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:31.018886 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01111d30-3f60-43e4-90db-e559a82a589a None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:31.021030 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 699/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 05:12:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:12:31.027405 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10254be-9e35-4c47-8e86-e3a94056d828 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:31.028150 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10254be-9e35-4c47-8e86-e3a94056d828 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:31.028286 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10254be-9e35-4c47-8e86-e3a94056d828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:31.100386 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b10254be-9e35-4c47-8e86-e3a94056d828 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:12:31.101572 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b10254be-9e35-4c47-8e86-e3a94056d828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:31.158295 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:31.207939 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 716/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 05:12:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:12:31.502144 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01111d30-3f60-43e4-90db-e559a82a589a None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:12:31.503693 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-01111d30-3f60-43e4-90db-e559a82a589a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:31.656574 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01111d30-3f60-43e4-90db-e559a82a589a None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:12:31.734558 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:31.778172 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc89b79-d365-4b3e-b541-05e1a81ca3d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:12:31.778776 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc89b79-d365-4b3e-b541-05e1a81ca3d5 service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:31.779047 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc89b79-d365-4b3e-b541-05e1a81ca3d5 service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:31.779305 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc89b79-d365-4b3e-b541-05e1a81ca3d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:31.780899 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-cfc89b79-d365-4b3e-b541-05e1a81ca3d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:31.880164 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc89b79-d365-4b3e-b541-05e1a81ca3d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:12:31.881802 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc89b79-d365-4b3e-b541-05e1a81ca3d5 service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:12:31.902285 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 700/1416] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:12:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:12:32.791917 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302ee02-2e40-4ddb-af23-8cbb5f99e191 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:32.792866 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302ee02-2e40-4ddb-af23-8cbb5f99e191 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:32.792961 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302ee02-2e40-4ddb-af23-8cbb5f99e191 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:32.795278 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 717/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 05:12:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:12:32.800572 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579b18c-bcbb-4de2-b41e-fa3b47d885c0 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:32.801019 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579b18c-bcbb-4de2-b41e-fa3b47d885c0 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:32.801116 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579b18c-bcbb-4de2-b41e-fa3b47d885c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:32.882361 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-9579b18c-bcbb-4de2-b41e-fa3b47d885c0 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:12:32.883586 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-9579b18c-bcbb-4de2-b41e-fa3b47d885c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:32.914558 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:32.988181 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 701/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 05:12:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:12:33.153750 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e302ee02-2e40-4ddb-af23-8cbb5f99e191 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:12:33.155378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e302ee02-2e40-4ddb-af23-8cbb5f99e191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:33.304814 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e302ee02-2e40-4ddb-af23-8cbb5f99e191 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:12:33.345384 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:33.421698 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eae9fb-c7d4-4b43-9718-bbe6edd4cdfc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:12:33.421698 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eae9fb-c7d4-4b43-9718-bbe6edd4cdfc service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:33.422095 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eae9fb-c7d4-4b43-9718-bbe6edd4cdfc service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:33.422095 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eae9fb-c7d4-4b43-9718-bbe6edd4cdfc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:33.423669 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-09eae9fb-c7d4-4b43-9718-bbe6edd4cdfc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:33.522223 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eae9fb-c7d4-4b43-9718-bbe6edd4cdfc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:12:33.524560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eae9fb-c7d4-4b43-9718-bbe6edd4cdfc service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:12:33.549393 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 718/1419] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:12:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:12:34.423531 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787eabe-83a3-43f5-ad15-d998705a67a1 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:34.424105 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787eabe-83a3-43f5-ad15-d998705a67a1 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:34.424105 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787eabe-83a3-43f5-ad15-d998705a67a1 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:34.425388 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 702/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 05:12:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:12:34.432130 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4653317b-0240-475a-961d-231df89697bc None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:34.432130 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4653317b-0240-475a-961d-231df89697bc None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:34.433642 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4653317b-0240-475a-961d-231df89697bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:34.488827 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:34.513627 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-4653317b-0240-475a-961d-231df89697bc None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:12:34.514885 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4653317b-0240-475a-961d-231df89697bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:34.620167 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 719/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 05:12:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:12:34.773258 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f787eabe-83a3-43f5-ad15-d998705a67a1 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:12:34.775748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-f787eabe-83a3-43f5-ad15-d998705a67a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:34.940332 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f787eabe-83a3-43f5-ad15-d998705a67a1 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:12:34.952803 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:35.063937 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c940dd7-2d82-4e27-ad8e-8b2d549624a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:12:35.064471 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c940dd7-2d82-4e27-ad8e-8b2d549624a8 service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:35.064742 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c940dd7-2d82-4e27-ad8e-8b2d549624a8 service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:35.065013 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c940dd7-2d82-4e27-ad8e-8b2d549624a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:35.066665 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-5c940dd7-2d82-4e27-ad8e-8b2d549624a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:35.165136 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c940dd7-2d82-4e27-ad8e-8b2d549624a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:12:35.166823 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c940dd7-2d82-4e27-ad8e-8b2d549624a8 service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:12:35.185987 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 703/1422] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:12:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:12:36.056208 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab3eb58-aef0-4714-b824-55430b849a0e None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:36.056699 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab3eb58-aef0-4714-b824-55430b849a0e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:36.056820 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab3eb58-aef0-4714-b824-55430b849a0e None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:36.058200 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 720/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 05:12:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:12:36.064291 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b717e-f34e-479e-b141-99e5bc0884a4 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:36.064757 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b717e-f34e-479e-b141-99e5bc0884a4 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:36.064851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b717e-f34e-479e-b141-99e5bc0884a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:36.095155 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:36.155049 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-c76b717e-f34e-479e-b141-99e5bc0884a4 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:12:36.156259 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c76b717e-f34e-479e-b141-99e5bc0884a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:36.258726 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 704/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 05:12:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:12:36.422104 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab3eb58-aef0-4714-b824-55430b849a0e None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:12:36.424242 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9ab3eb58-aef0-4714-b824-55430b849a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:36.561256 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:36.599836 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab3eb58-aef0-4714-b824-55430b849a0e None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:12:36.728036 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0154a3b7-2772-4ea2-81d7-7723fa97bfc6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:12:36.728562 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154a3b7-2772-4ea2-81d7-7723fa97bfc6 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:36.728835 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154a3b7-2772-4ea2-81d7-7723fa97bfc6 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:36.729096 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154a3b7-2772-4ea2-81d7-7723fa97bfc6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:36.730719 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0154a3b7-2772-4ea2-81d7-7723fa97bfc6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:36.807488 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:36.821234 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0154a3b7-2772-4ea2-81d7-7723fa97bfc6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:12:36.822859 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0154a3b7-2772-4ea2-81d7-7723fa97bfc6 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:12:36.841023 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 721/1425] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:12:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:12:37.681513 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ad68d-1f88-4136-8ea5-01e6defef4ed None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:37.682038 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ad68d-1f88-4136-8ea5-01e6defef4ed None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:37.682038 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ad68d-1f88-4136-8ea5-01e6defef4ed None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:37.683505 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 705/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 05:12:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:12:37.689496 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d285e2-79d1-4cac-89be-069fe9f0424c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:37.690221 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d285e2-79d1-4cac-89be-069fe9f0424c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:37.690314 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d285e2-79d1-4cac-89be-069fe9f0424c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:37.772320 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a4d285e2-79d1-4cac-89be-069fe9f0424c None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:12:37.773511 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a4d285e2-79d1-4cac-89be-069fe9f0424c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:37.883160 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 722/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 05:12:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:12:38.032462 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ad68d-1f88-4136-8ea5-01e6defef4ed None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:12:38.034943 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-6c0ad68d-1f88-4136-8ea5-01e6defef4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:38.133071 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:38.197742 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ad68d-1f88-4136-8ea5-01e6defef4ed None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:12:38.320759 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5f3d98-0cdc-4242-baf6-b75f7a14629b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:12:38.321288 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f3d98-0cdc-4242-baf6-b75f7a14629b service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:38.321562 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f3d98-0cdc-4242-baf6-b75f7a14629b service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:38.321826 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f3d98-0cdc-4242-baf6-b75f7a14629b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:38.324418 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-6b5f3d98-0cdc-4242-baf6-b75f7a14629b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:38.381699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:38.427287 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5f3d98-0cdc-4242-baf6-b75f7a14629b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:12:38.428927 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5f3d98-0cdc-4242-baf6-b75f7a14629b service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:12:38.447760 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 706/1428] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:12:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:12:39.273885 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77146b1-d045-4385-a612-2d5fc61bc997 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:39.274865 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77146b1-d045-4385-a612-2d5fc61bc997 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:39.274865 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77146b1-d045-4385-a612-2d5fc61bc997 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:39.277165 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 723/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 05:12:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:12:39.282918 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85202c-86e4-43a2-a247-66bec4077841 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:39.283700 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85202c-86e4-43a2-a247-66bec4077841 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:39.283839 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85202c-86e4-43a2-a247-66bec4077841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:39.368966 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-dc85202c-86e4-43a2-a247-66bec4077841 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:12:39.370217 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-dc85202c-86e4-43a2-a247-66bec4077841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:39.470314 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 707/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 05:12:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:12:39.624758 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77146b1-d045-4385-a612-2d5fc61bc997 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:12:39.627112 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f77146b1-d045-4385-a612-2d5fc61bc997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:39.685910 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:39.794665 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77146b1-d045-4385-a612-2d5fc61bc997 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:12:39.921837 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da56de-4061-45a1-b604-4429cb22c29c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:12:39.922406 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da56de-4061-45a1-b604-4429cb22c29c service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:39.922688 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da56de-4061-45a1-b604-4429cb22c29c service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:39.922950 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da56de-4061-45a1-b604-4429cb22c29c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:39.924647 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-07da56de-4061-45a1-b604-4429cb22c29c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:39.931770 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:40.019049 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07da56de-4061-45a1-b604-4429cb22c29c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:12:40.020635 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07da56de-4061-45a1-b604-4429cb22c29c service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:12:40.040937 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 724/1431] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:12:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:12:40.868811 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2dabb5-f6e9-426a-97ce-c42c892d03e4 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:40.869527 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2dabb5-f6e9-426a-97ce-c42c892d03e4 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:40.869527 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2dabb5-f6e9-426a-97ce-c42c892d03e4 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:40.870856 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 708/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 05:12:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 05:12:40.879511 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa72921-6a72-4004-bf14-da07df97ef04 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:40.880299 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa72921-6a72-4004-bf14-da07df97ef04 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:40.880487 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa72921-6a72-4004-bf14-da07df97ef04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:40.968263 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-8fa72921-6a72-4004-bf14-da07df97ef04 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:12:40.969877 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8fa72921-6a72-4004-bf14-da07df97ef04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:41.043616 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:41.078142 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 725/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 05:12:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:12:41.228047 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2dabb5-f6e9-426a-97ce-c42c892d03e4 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:12:41.229494 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-db2dabb5-f6e9-426a-97ce-c42c892d03e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:41.249468 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:41.383044 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2dabb5-f6e9-426a-97ce-c42c892d03e4 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:12:41.477359 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:41.506279 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40be125-5470-48a1-9389-16a1c1091aea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:12:41.506882 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40be125-5470-48a1-9389-16a1c1091aea service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:41.507156 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40be125-5470-48a1-9389-16a1c1091aea service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:41.507418 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40be125-5470-48a1-9389-16a1c1091aea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:41.508999 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b40be125-5470-48a1-9389-16a1c1091aea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:41.600481 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40be125-5470-48a1-9389-16a1c1091aea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:12:41.602114 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40be125-5470-48a1-9389-16a1c1091aea service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:12:41.620306 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 709/1434] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:12:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:12:42.412537 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b0889-d4e4-4e3c-820e-0fc6e36bdd51 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:42.413437 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b0889-d4e4-4e3c-820e-0fc6e36bdd51 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:42.413437 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b0889-d4e4-4e3c-820e-0fc6e36bdd51 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:42.415588 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 726/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 05:12:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:12:42.421725 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723a7ab-ca2d-4b60-aa92-d448a3b55b9a None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:42.422442 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723a7ab-ca2d-4b60-aa92-d448a3b55b9a None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:42.422622 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723a7ab-ca2d-4b60-aa92-d448a3b55b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:42.512859 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-7723a7ab-ca2d-4b60-aa92-d448a3b55b9a None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:12:42.514072 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-7723a7ab-ca2d-4b60-aa92-d448a3b55b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:42.557177 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:42.615481 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 710/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 05:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:12:42.775587 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b0889-d4e4-4e3c-820e-0fc6e36bdd51 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:12:42.777240 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-373b0889-d4e4-4e3c-820e-0fc6e36bdd51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:42.931104 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b0889-d4e4-4e3c-820e-0fc6e36bdd51 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:12:42.990048 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:43.049329 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf7e0c-c682-4bef-8e8c-915ee4d23ff3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:12:43.049862 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf7e0c-c682-4bef-8e8c-915ee4d23ff3 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:43.050170 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf7e0c-c682-4bef-8e8c-915ee4d23ff3 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:43.050437 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf7e0c-c682-4bef-8e8c-915ee4d23ff3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:43.052043 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f4bf7e0c-c682-4bef-8e8c-915ee4d23ff3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:43.142890 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bf7e0c-c682-4bef-8e8c-915ee4d23ff3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:12:43.144489 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bf7e0c-c682-4bef-8e8c-915ee4d23ff3 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:12:43.163155 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 727/1437] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:12:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:12:43.962657 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be25541-8215-4b13-887b-f95ba48ce824 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:43.963211 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be25541-8215-4b13-887b-f95ba48ce824 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:43.963311 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be25541-8215-4b13-887b-f95ba48ce824 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:43.964643 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 711/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 05:12:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:12:43.972427 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe0b369-5f70-4902-a91e-893050cb6570 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:43.972839 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe0b369-5f70-4902-a91e-893050cb6570 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:43.972993 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe0b369-5f70-4902-a91e-893050cb6570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:44.061234 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-9fe0b369-5f70-4902-a91e-893050cb6570 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:12:44.062502 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9fe0b369-5f70-4902-a91e-893050cb6570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:44.064648 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:44.171958 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 728/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 05:12:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:12:44.324774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be25541-8215-4b13-887b-f95ba48ce824 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:12:44.327304 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-1be25541-8215-4b13-887b-f95ba48ce824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:44.492975 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be25541-8215-4b13-887b-f95ba48ce824 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:12:44.521593 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:44.612282 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ea5b66-fb27-405a-b248-12670f85e9f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:12:44.612936 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea5b66-fb27-405a-b248-12670f85e9f0 service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:44.613251 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea5b66-fb27-405a-b248-12670f85e9f0 service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:44.613573 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea5b66-fb27-405a-b248-12670f85e9f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:44.616122 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-04ea5b66-fb27-405a-b248-12670f85e9f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:44.720254 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ea5b66-fb27-405a-b248-12670f85e9f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:12:44.721975 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ea5b66-fb27-405a-b248-12670f85e9f0 service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:12:44.741036 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 712/1440] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:12:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:12:46.545680 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd3d95-a841-402d-8463-f372afcec69d None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:46.546270 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd3d95-a841-402d-8463-f372afcec69d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:46.546270 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd3d95-a841-402d-8463-f372afcec69d None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:46.547523 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 729/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 05:12:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:12:46.553612 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917deeab-a8ed-49f6-9292-248f381888e1 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:46.554425 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917deeab-a8ed-49f6-9292-248f381888e1 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:46.554541 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917deeab-a8ed-49f6-9292-248f381888e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:46.599121 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:46.626394 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-917deeab-a8ed-49f6-9292-248f381888e1 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:12:46.627606 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-917deeab-a8ed-49f6-9292-248f381888e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:46.734395 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 713/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 05:12:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:12:47.031857 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd3d95-a841-402d-8463-f372afcec69d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:12:47.033431 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-dacd3d95-a841-402d-8463-f372afcec69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:47.148961 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd3d95-a841-402d-8463-f372afcec69d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:12:47.183145 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:47.236236 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b09d597-7b04-4a5c-8419-b733a14380d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:12:47.237027 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b09d597-7b04-4a5c-8419-b733a14380d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:47.237372 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b09d597-7b04-4a5c-8419-b733a14380d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:47.237749 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b09d597-7b04-4a5c-8419-b733a14380d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:47.240274 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9b09d597-7b04-4a5c-8419-b733a14380d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:47.355541 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b09d597-7b04-4a5c-8419-b733a14380d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:12:47.356951 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b09d597-7b04-4a5c-8419-b733a14380d6 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:12:47.376549 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 730/1443] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:12:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:12:49.448265 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87455b7d-0746-4b8a-a035-67d590037c5d None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:49.449145 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87455b7d-0746-4b8a-a035-67d590037c5d None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:49.449145 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87455b7d-0746-4b8a-a035-67d590037c5d None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:49.451434 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 714/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 05:12:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:12:49.460675 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a093e-7938-4141-a133-3a5b2927eb5f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:49.461390 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a093e-7938-4141-a133-3a5b2927eb5f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:49.461538 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a093e-7938-4141-a133-3a5b2927eb5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:49.532292 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-868a093e-7938-4141-a133-3a5b2927eb5f None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:12:49.533788 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-868a093e-7938-4141-a133-3a5b2927eb5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:49.554768 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:49.644300 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 731/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 05:12:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:12:49.923611 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87455b7d-0746-4b8a-a035-67d590037c5d None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:12:49.925724 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-87455b7d-0746-4b8a-a035-67d590037c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:50.043888 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87455b7d-0746-4b8a-a035-67d590037c5d None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:12:50.056933 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:50.122532 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d8e7a-64f6-452c-9d03-0cfcae78fd68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:12:50.123126 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d8e7a-64f6-452c-9d03-0cfcae78fd68 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:50.123394 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d8e7a-64f6-452c-9d03-0cfcae78fd68 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:50.123655 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d8e7a-64f6-452c-9d03-0cfcae78fd68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:50.125343 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-4a8d8e7a-64f6-452c-9d03-0cfcae78fd68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:50.236030 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d8e7a-64f6-452c-9d03-0cfcae78fd68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:12:50.237385 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d8e7a-64f6-452c-9d03-0cfcae78fd68 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:12:50.255566 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 715/1446] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:12:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:12:52.451294 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8fd743-7e3b-492d-99da-288d986b0a4e None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:52.451861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8fd743-7e3b-492d-99da-288d986b0a4e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:52.451861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8fd743-7e3b-492d-99da-288d986b0a4e None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:52.453212 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 732/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 05:12:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:12:52.459649 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f2f94-1e23-41af-89a1-0de83ac654b0 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:52.460418 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f2f94-1e23-41af-89a1-0de83ac654b0 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:52.460598 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f2f94-1e23-41af-89a1-0de83ac654b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:52.515956 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:52.534521 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-274f2f94-1e23-41af-89a1-0de83ac654b0 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:12:52.535754 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-274f2f94-1e23-41af-89a1-0de83ac654b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:52.635314 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 716/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 05:12:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:12:52.914815 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8fd743-7e3b-492d-99da-288d986b0a4e None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:12:52.917359 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ef8fd743-7e3b-492d-99da-288d986b0a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:53.035830 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8fd743-7e3b-492d-99da-288d986b0a4e None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:12:53.078382 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:53.113490 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b44339-fbfe-4708-ab6f-30936f979387 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:12:53.114119 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b44339-fbfe-4708-ab6f-30936f979387 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:53.114390 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b44339-fbfe-4708-ab6f-30936f979387 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:53.114653 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b44339-fbfe-4708-ab6f-30936f979387 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:53.116227 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-74b44339-fbfe-4708-ab6f-30936f979387 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:53.232056 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b44339-fbfe-4708-ab6f-30936f979387 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:12:53.233721 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b44339-fbfe-4708-ab6f-30936f979387 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:12:53.253611 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 733/1449] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:12:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:12:55.273869 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b286ea9-d525-4186-b460-85a0ce066056 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:55.274842 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b286ea9-d525-4186-b460-85a0ce066056 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:55.274842 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b286ea9-d525-4186-b460-85a0ce066056 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:55.277042 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 717/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 05:12:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:12:55.283534 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7f1df-0c5e-4f5f-81b4-a624f357a0b1 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:55.284334 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7f1df-0c5e-4f5f-81b4-a624f357a0b1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:55.284334 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7f1df-0c5e-4f5f-81b4-a624f357a0b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:55.358767 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-17e7f1df-0c5e-4f5f-81b4-a624f357a0b1 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:12:55.359992 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-17e7f1df-0c5e-4f5f-81b4-a624f357a0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:55.381991 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:55.460619 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 734/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 05:12:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:12:55.752098 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b286ea9-d525-4186-b460-85a0ce066056 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:12:55.754635 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-4b286ea9-d525-4186-b460-85a0ce066056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:55.864993 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b286ea9-d525-4186-b460-85a0ce066056 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:12:55.884179 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:55.954149 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b385dca9-6b1f-4be6-a475-bf58017ab7d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:12:55.954647 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385dca9-6b1f-4be6-a475-bf58017ab7d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:55.954951 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385dca9-6b1f-4be6-a475-bf58017ab7d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:55.955209 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385dca9-6b1f-4be6-a475-bf58017ab7d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:55.956996 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b385dca9-6b1f-4be6-a475-bf58017ab7d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:56.077733 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b385dca9-6b1f-4be6-a475-bf58017ab7d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:12:56.079239 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b385dca9-6b1f-4be6-a475-bf58017ab7d0 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:12:56.097609 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 718/1452] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:12:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:12:58.061484 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39d8f7-648e-46ba-9054-fa7aaf86aacb None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:58.061976 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39d8f7-648e-46ba-9054-fa7aaf86aacb None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:58.062098 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39d8f7-648e-46ba-9054-fa7aaf86aacb None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:58.064235 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 735/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 05:12:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:12:58.073065 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc8ae9-0655-4b29-8b51-4925e66566f1 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:58.073494 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc8ae9-0655-4b29-8b51-4925e66566f1 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:58.073641 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc8ae9-0655-4b29-8b51-4925e66566f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:58.141352 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:58.144859 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-3ffc8ae9-0655-4b29-8b51-4925e66566f1 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:12:58.147195 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3ffc8ae9-0655-4b29-8b51-4925e66566f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:58.265468 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 719/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 05:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:12:58.560416 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e39d8f7-648e-46ba-9054-fa7aaf86aacb None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:12:58.561954 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2e39d8f7-648e-46ba-9054-fa7aaf86aacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:58.681787 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e39d8f7-648e-46ba-9054-fa7aaf86aacb None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:12:58.734587 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:12:58.760012 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d263d34-933c-491e-9f7f-a30839dfca25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:12:58.760624 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d263d34-933c-491e-9f7f-a30839dfca25 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:12:58.760903 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d263d34-933c-491e-9f7f-a30839dfca25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:12:58.761171 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d263d34-933c-491e-9f7f-a30839dfca25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:12:58.762892 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6d263d34-933c-491e-9f7f-a30839dfca25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:12:58.876026 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d263d34-933c-491e-9f7f-a30839dfca25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:12:58.877385 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d263d34-933c-491e-9f7f-a30839dfca25 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:12:58.895768 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 736/1455] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:12:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:13:00.836000 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d5172-02c1-4aac-a618-c88ad521feff None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:13:00.836607 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d5172-02c1-4aac-a618-c88ad521feff None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:13:00.836607 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d5172-02c1-4aac-a618-c88ad521feff None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:13:00.837922 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 720/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 05:13:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:13:00.844427 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85954b80-d15f-4d19-a08c-56b6b73f6d67 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:13:00.845191 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85954b80-d15f-4d19-a08c-56b6b73f6d67 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:13:00.845340 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85954b80-d15f-4d19-a08c-56b6b73f6d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:13:00.927823 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-85954b80-d15f-4d19-a08c-56b6b73f6d67 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:13:00.928943 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-85954b80-d15f-4d19-a08c-56b6b73f6d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:13:00.961763 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:13:01.030894 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 737/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 05:13:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:13:01.320324 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d5172-02c1-4aac-a618-c88ad521feff None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:13:01.321855 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-fd9d5172-02c1-4aac-a618-c88ad521feff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:13:01.443758 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d5172-02c1-4aac-a618-c88ad521feff None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:13:01.472583 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:13:01.520030 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff19dce-26d7-44d2-991e-86643147acc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:13:01.520637 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff19dce-26d7-44d2-991e-86643147acc9 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:13:01.520912 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff19dce-26d7-44d2-991e-86643147acc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:13:01.521172 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff19dce-26d7-44d2-991e-86643147acc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:13:01.522803 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-2ff19dce-26d7-44d2-991e-86643147acc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:13:01.632147 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff19dce-26d7-44d2-991e-86643147acc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:13:01.633503 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff19dce-26d7-44d2-991e-86643147acc9 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:13:01.652994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 721/1458] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:13:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 05:13:03.595778 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107c198-7a9d-43bb-85b0-e221ad566402 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:13:03.596525 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107c198-7a9d-43bb-85b0-e221ad566402 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:13:03.596525 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107c198-7a9d-43bb-85b0-e221ad566402 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:13:03.598665 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 738/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 05:13:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:13:03.607814 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3335b9-5ee1-4419-9609-fe95cc05f118 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:13:03.608556 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3335b9-5ee1-4419-9609-fe95cc05f118 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:13:03.608673 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3335b9-5ee1-4419-9609-fe95cc05f118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:13:03.683115 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-1b3335b9-5ee1-4419-9609-fe95cc05f118 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:13:03.684403 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-1b3335b9-5ee1-4419-9609-fe95cc05f118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:13:03.692839 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:13:03.783688 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 722/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 05:13:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:13:04.076409 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6107c198-7a9d-43bb-85b0-e221ad566402 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:13:04.077981 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6107c198-7a9d-43bb-85b0-e221ad566402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:13:04.183796 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6107c198-7a9d-43bb-85b0-e221ad566402 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:13:04.243982 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:13:04.264926 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95245b53-f05d-43d3-ad41-0dd6b22092a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:13:04.265534 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95245b53-f05d-43d3-ad41-0dd6b22092a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:13:04.265814 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95245b53-f05d-43d3-ad41-0dd6b22092a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:13:04.266112 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95245b53-f05d-43d3-ad41-0dd6b22092a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:13:04.267717 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-95245b53-f05d-43d3-ad41-0dd6b22092a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:13:04.393344 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95245b53-f05d-43d3-ad41-0dd6b22092a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:13:04.394749 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95245b53-f05d-43d3-ad41-0dd6b22092a0 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:13:04.413238 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 739/1461] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:13:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:13:06.348107 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d777a-f385-4ec0-ad82-9dd68ba73d70 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:13:06.348658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d777a-f385-4ec0-ad82-9dd68ba73d70 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:13:06.348658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d777a-f385-4ec0-ad82-9dd68ba73d70 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:13:06.350722 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 723/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 05:13:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:13:06.360064 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32a0b0-85d8-48e4-8086-acfd833b0093 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:13:06.360605 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32a0b0-85d8-48e4-8086-acfd833b0093 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:13:06.360605 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32a0b0-85d8-48e4-8086-acfd833b0093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:13:06.436581 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-bd32a0b0-85d8-48e4-8086-acfd833b0093 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:13:06.437770 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-bd32a0b0-85d8-48e4-8086-acfd833b0093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:13:06.477510 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:13:06.540805 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 740/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 05:13:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:13:06.826687 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d777a-f385-4ec0-ad82-9dd68ba73d70 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:13:06.829084 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-680d777a-f385-4ec0-ad82-9dd68ba73d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:13:06.940992 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d777a-f385-4ec0-ad82-9dd68ba73d70 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:13:06.988132 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:13:07.023928 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b830a-f15d-4316-a7cc-5b3d8b4769bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:13:07.024536 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b830a-f15d-4316-a7cc-5b3d8b4769bd service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:13:07.024808 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b830a-f15d-4316-a7cc-5b3d8b4769bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:13:07.025074 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b830a-f15d-4316-a7cc-5b3d8b4769bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:13:07.026694 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-d85b830a-f15d-4316-a7cc-5b3d8b4769bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:13:07.137379 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85b830a-f15d-4316-a7cc-5b3d8b4769bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:13:07.138861 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85b830a-f15d-4316-a7cc-5b3d8b4769bd service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:13:07.156836 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 724/1464] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:13:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:13:09.062040 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d6f89-4700-43f6-b6a3-e8dc64fd6da0 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:13:09.062630 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d6f89-4700-43f6-b6a3-e8dc64fd6da0 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:13:09.062630 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d6f89-4700-43f6-b6a3-e8dc64fd6da0 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:13:09.063970 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 741/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 05:13:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:13:09.070356 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e79556-b012-4e31-b4b7-a37df4a83a05 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:13:09.071030 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e79556-b012-4e31-b4b7-a37df4a83a05 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:13:09.071188 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e79556-b012-4e31-b4b7-a37df4a83a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:13:09.144315 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-28e79556-b012-4e31-b4b7-a37df4a83a05 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:13:09.145492 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-28e79556-b012-4e31-b4b7-a37df4a83a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:13:09.163852 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:13:09.245972 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 725/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 05:13:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:13:09.524609 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5d6f89-4700-43f6-b6a3-e8dc64fd6da0 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:13:09.526225 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8a5d6f89-4700-43f6-b6a3-e8dc64fd6da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:13:09.633636 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5d6f89-4700-43f6-b6a3-e8dc64fd6da0 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:13:09.709272 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:13:09.719109 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f05dd52-a941-4e57-946d-4eb082db79e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:13:09.719957 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05dd52-a941-4e57-946d-4eb082db79e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:13:09.720461 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05dd52-a941-4e57-946d-4eb082db79e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:13:09.720755 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05dd52-a941-4e57-946d-4eb082db79e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:13:09.722448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9f05dd52-a941-4e57-946d-4eb082db79e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:13:09.836042 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f05dd52-a941-4e57-946d-4eb082db79e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:13:09.837405 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f05dd52-a941-4e57-946d-4eb082db79e1 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:13:09.855609 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 742/1467] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:13:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:13:11.747586 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1db17e4-304d-416a-8592-a5c2d9c3cb1b None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:13:11.748219 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1db17e4-304d-416a-8592-a5c2d9c3cb1b None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:13:11.748219 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1db17e4-304d-416a-8592-a5c2d9c3cb1b None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:13:11.749483 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 726/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 05:13:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 05:13:11.756038 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c03e2e-04a3-41fa-8de1-d639b940b622 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:13:11.756478 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c03e2e-04a3-41fa-8de1-d639b940b622 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:13:11.756581 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c03e2e-04a3-41fa-8de1-d639b940b622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:13:11.830589 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-81c03e2e-04a3-41fa-8de1-d639b940b622 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:13:11.831720 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-81c03e2e-04a3-41fa-8de1-d639b940b622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:13:11.883218 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:13:11.933844 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 743/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 05:13:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:13:12.252310 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1db17e4-304d-416a-8592-a5c2d9c3cb1b None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:13:12.254661 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a1db17e4-304d-416a-8592-a5c2d9c3cb1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:13:12.385141 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1db17e4-304d-416a-8592-a5c2d9c3cb1b None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:13:12.439621 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:13:12.469422 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ddeffc-0584-49c6-8fcc-24e185808f3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:13:12.470043 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ddeffc-0584-49c6-8fcc-24e185808f3d service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:13:12.470333 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ddeffc-0584-49c6-8fcc-24e185808f3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:13:12.470587 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ddeffc-0584-49c6-8fcc-24e185808f3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:13:12.472206 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-87ddeffc-0584-49c6-8fcc-24e185808f3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:13:12.604695 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ddeffc-0584-49c6-8fcc-24e185808f3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:13:12.606244 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ddeffc-0584-49c6-8fcc-24e185808f3d service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:13:12.626703 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 727/1470] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:13:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:13:21.932110 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=22 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:13:22.624008 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=66 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:13:31.926767 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40af307-91a3-47f0-a38d-8d75a01dd811 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:13:31.927323 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40af307-91a3-47f0-a38d-8d75a01dd811 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:13:31.927323 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40af307-91a3-47f0-a38d-8d75a01dd811 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:13:31.928596 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 744/1471] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:13:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:13:31.934131 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d2e84-0eb7-4738-9ffd-ecfa502a2571 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:13:31.934595 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d2e84-0eb7-4738-9ffd-ecfa502a2571 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:13:31.934705 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d2e84-0eb7-4738-9ffd-ecfa502a2571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:13:32.005432 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-3b9d2e84-0eb7-4738-9ffd-ecfa502a2571 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:13:32.006687 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3b9d2e84-0eb7-4738-9ffd-ecfa502a2571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:13:32.115880 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 728/1472] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:13:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:13:32.407772 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3559c-87fa-4a4d-9363-f98bf5fe4a52 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:13:32.408058 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3559c-87fa-4a4d-9363-f98bf5fe4a52 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:13:32.408329 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3559c-87fa-4a4d-9363-f98bf5fe4a52 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:13:32.409684 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 745/1473] 10.222.0.36 () {54 vars in 967 bytes} [Wed Aug 31 05:13:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:13:32.414661 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a25388-cbd9-4971-a735-4990ca61c6f6 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:13:32.414941 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a25388-cbd9-4971-a735-4990ca61c6f6 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:13:32.415205 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a25388-cbd9-4971-a735-4990ca61c6f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:13:32.491950 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-d8a25388-cbd9-4971-a735-4990ca61c6f6 None None] MFA Rules not processed for user `c08e6024b0c8420aa274653c2a916edd`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:13:32.493200 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-d8a25388-cbd9-4971-a735-4990ca61c6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:13:32.515635 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:13:32.578807 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 729/1474] 10.222.0.36 () {58 vars in 1064 bytes} [Wed Aug 31 05:13:32 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:13:32.586636 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c3559c-87fa-4a4d-9363-f98bf5fe4a52 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:13:32.588192 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-34c3559c-87fa-4a4d-9363-f98bf5fe4a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:13:32.698367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c3559c-87fa-4a4d-9363-f98bf5fe4a52 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:13:32.782376 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25086d8-ce15-411b-8931-7905354825e1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08e6024b0c8420aa274653c2a916edd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:13:32.783247 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25086d8-ce15-411b-8931-7905354825e1 service glance] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:13:32.783583 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25086d8-ce15-411b-8931-7905354825e1 service glance] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:13:32.783957 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25086d8-ce15-411b-8931-7905354825e1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:13:32.786575 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b25086d8-ce15-411b-8931-7905354825e1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:13:32.848772 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:13:32.905604 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25086d8-ce15-411b-8931-7905354825e1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:13:32.907000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25086d8-ce15-411b-8931-7905354825e1 service glance] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:13:32.925570 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 746/1475] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:13:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:13:34.902910 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963c374-7ee8-4353-88cc-a4c358f60137 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:13:34.903728 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963c374-7ee8-4353-88cc-a4c358f60137 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:13:34.903728 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963c374-7ee8-4353-88cc-a4c358f60137 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:13:34.905960 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 730/1476] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:13:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:13:34.914973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f783f8-890b-4c21-8b51-b1e16ac6e360 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:13:34.915679 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f783f8-890b-4c21-8b51-b1e16ac6e360 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:13:34.915867 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f783f8-890b-4c21-8b51-b1e16ac6e360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:13:35.117544 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-69f783f8-890b-4c21-8b51-b1e16ac6e360 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:13:35.118803 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-69f783f8-890b-4c21-8b51-b1e16ac6e360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:13:35.216398 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 747/1477] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:13:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:13:35.490434 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4963c374-7ee8-4353-88cc-a4c358f60137 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:13:35.491887 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-4963c374-7ee8-4353-88cc-a4c358f60137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:13:35.623168 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4963c374-7ee8-4353-88cc-a4c358f60137 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:13:35.673176 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:13:35.723065 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d0747-2de3-4947-8dfe-74670ffd4213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:13:35.723672 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d0747-2de3-4947-8dfe-74670ffd4213 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:13:35.723949 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d0747-2de3-4947-8dfe-74670ffd4213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:13:35.724211 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d0747-2de3-4947-8dfe-74670ffd4213 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:13:35.724875 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6d0747-2de3-4947-8dfe-74670ffd4213 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:13:35.726713 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: WARNING py.warnings [None req-6b6d0747-2de3-4947-8dfe-74670ffd4213 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 05:13:35.726713 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: warnings.warn(msg) Aug 31 05:13:35.726713 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: Aug 31 05:13:35.727861 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6d0747-2de3-4947-8dfe-74670ffd4213 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:13:35.747952 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 731/1478] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Aug 31 05:13:35 2022] GET /identity/v3/projects => generated 2553 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:13:37.681628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff53595-ccd6-49b6-86c1-f6b27c7dc85c None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:13:37.682178 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff53595-ccd6-49b6-86c1-f6b27c7dc85c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:13:37.682178 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff53595-ccd6-49b6-86c1-f6b27c7dc85c None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:13:37.684266 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 748/1479] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:13:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:13:37.692782 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f15c75-813f-4910-bd0d-187b1a0c9840 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:13:37.693223 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f15c75-813f-4910-bd0d-187b1a0c9840 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:13:37.693422 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f15c75-813f-4910-bd0d-187b1a0c9840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:13:37.767599 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-d3f15c75-813f-4910-bd0d-187b1a0c9840 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:13:37.768842 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-d3f15c75-813f-4910-bd0d-187b1a0c9840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:13:37.870740 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 732/1480] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:13:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:13:38.163240 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff53595-ccd6-49b6-86c1-f6b27c7dc85c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:13:38.164756 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8ff53595-ccd6-49b6-86c1-f6b27c7dc85c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:13:38.174934 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:13:38.319927 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff53595-ccd6-49b6-86c1-f6b27c7dc85c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:13:38.404601 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:13:38.442058 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bb6435-deeb-48bb-b378-e5f34247c9a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:13:38.442665 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb6435-deeb-48bb-b378-e5f34247c9a3 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:13:38.442938 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb6435-deeb-48bb-b378-e5f34247c9a3 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:13:38.443200 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb6435-deeb-48bb-b378-e5f34247c9a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:13:38.444773 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c6bb6435-deeb-48bb-b378-e5f34247c9a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:13:38.538444 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bb6435-deeb-48bb-b378-e5f34247c9a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:13:38.540039 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bb6435-deeb-48bb-b378-e5f34247c9a3 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:13:38.558233 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 749/1481] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:13:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:13:40.650600 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78381af-db8d-415b-8fe4-8d3d0c81ec5f None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:13:40.651083 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78381af-db8d-415b-8fe4-8d3d0c81ec5f None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:13:40.651083 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78381af-db8d-415b-8fe4-8d3d0c81ec5f None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:13:40.652492 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 733/1482] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:13:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:13:40.658158 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f3f5b6-6528-4839-811e-98b8e8c8c01e None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:13:40.658631 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f3f5b6-6528-4839-811e-98b8e8c8c01e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:13:40.658737 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f3f5b6-6528-4839-811e-98b8e8c8c01e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:13:40.731068 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-60f3f5b6-6528-4839-811e-98b8e8c8c01e None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:13:40.732366 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-60f3f5b6-6528-4839-811e-98b8e8c8c01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:13:40.788737 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:13:40.842977 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 750/1483] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:13:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:13:41.129319 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78381af-db8d-415b-8fe4-8d3d0c81ec5f None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:13:41.131874 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a78381af-db8d-415b-8fe4-8d3d0c81ec5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:13:41.168181 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:13:41.261580 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78381af-db8d-415b-8fe4-8d3d0c81ec5f None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:13:41.337333 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce40c00-756d-4931-b3b4-19c8ec1c4193 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:13:41.337929 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce40c00-756d-4931-b3b4-19c8ec1c4193 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:13:41.338233 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce40c00-756d-4931-b3b4-19c8ec1c4193 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:13:41.338495 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce40c00-756d-4931-b3b4-19c8ec1c4193 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:13:41.340116 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-bce40c00-756d-4931-b3b4-19c8ec1c4193 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:13:41.420525 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:13:41.458940 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce40c00-756d-4931-b3b4-19c8ec1c4193 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:13:41.460392 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce40c00-756d-4931-b3b4-19c8ec1c4193 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:13:41.478658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 734/1484] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:13:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:13:43.429173 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd8f823-f34e-4731-851c-28486c8bc529 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:13:43.430107 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd8f823-f34e-4731-851c-28486c8bc529 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:13:43.430107 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd8f823-f34e-4731-851c-28486c8bc529 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:13:43.432196 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 751/1485] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:13:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:13:43.439527 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d8a1a2-52ee-4d99-8ce4-72eb72eb3e17 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:13:43.439998 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d8a1a2-52ee-4d99-8ce4-72eb72eb3e17 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:13:43.440101 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d8a1a2-52ee-4d99-8ce4-72eb72eb3e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:13:43.515918 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-c3d8a1a2-52ee-4d99-8ce4-72eb72eb3e17 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:13:43.517117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c3d8a1a2-52ee-4d99-8ce4-72eb72eb3e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:13:43.626871 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 735/1486] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:13:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:13:43.919594 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd8f823-f34e-4731-851c-28486c8bc529 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:13:43.921904 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4bd8f823-f34e-4731-851c-28486c8bc529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:13:44.049666 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd8f823-f34e-4731-851c-28486c8bc529 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:13:44.148727 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:13:44.150796 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a790970-92ea-4207-9708-fc97be3e1a38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:13:44.151437 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a790970-92ea-4207-9708-fc97be3e1a38 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:13:44.151980 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a790970-92ea-4207-9708-fc97be3e1a38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:13:44.152302 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a790970-92ea-4207-9708-fc97be3e1a38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:13:44.154683 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8a790970-92ea-4207-9708-fc97be3e1a38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:13:44.274762 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a790970-92ea-4207-9708-fc97be3e1a38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:13:44.276191 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a790970-92ea-4207-9708-fc97be3e1a38 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:13:44.295851 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 752/1487] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:13:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:13:46.347266 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be8fdb-f627-419e-88da-ee16e4366262 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:13:46.347795 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be8fdb-f627-419e-88da-ee16e4366262 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:13:46.347795 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be8fdb-f627-419e-88da-ee16e4366262 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:13:46.349166 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 736/1488] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:13:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:13:46.354628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e81af-d87b-4a3d-92e4-dce9962b7ba1 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:13:46.355086 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e81af-d87b-4a3d-92e4-dce9962b7ba1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:13:46.355184 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e81af-d87b-4a3d-92e4-dce9962b7ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:13:46.424857 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-215e81af-d87b-4a3d-92e4-dce9962b7ba1 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:13:46.426084 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-215e81af-d87b-4a3d-92e4-dce9962b7ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:13:46.486361 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:13:46.533156 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 753/1489] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:13:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:13:46.825864 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be8fdb-f627-419e-88da-ee16e4366262 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:13:46.827439 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-20be8fdb-f627-419e-88da-ee16e4366262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:13:46.868585 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:13:46.942476 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be8fdb-f627-419e-88da-ee16e4366262 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:13:47.027519 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a6bbd-f7e7-42f8-89f5-2a9a04cf152c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:13:47.028100 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a6bbd-f7e7-42f8-89f5-2a9a04cf152c service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:13:47.028372 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a6bbd-f7e7-42f8-89f5-2a9a04cf152c service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:13:47.028660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a6bbd-f7e7-42f8-89f5-2a9a04cf152c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:13:47.030296 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b50a6bbd-f7e7-42f8-89f5-2a9a04cf152c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:13:47.115597 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:13:47.149741 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50a6bbd-f7e7-42f8-89f5-2a9a04cf152c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:13:47.151243 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50a6bbd-f7e7-42f8-89f5-2a9a04cf152c service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:13:47.169634 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 737/1490] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:13:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:13:49.357055 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe36e1d-4f5e-40b3-af97-cafe14e42936 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:13:49.357638 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe36e1d-4f5e-40b3-af97-cafe14e42936 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:13:49.357638 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe36e1d-4f5e-40b3-af97-cafe14e42936 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:13:49.358936 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 754/1491] 10.222.0.36 () {54 vars in 927 bytes} [Wed Aug 31 05:13:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:13:49.366815 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0552de-f59c-4246-8ae1-1425028e5b87 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:13:49.367470 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0552de-f59c-4246-8ae1-1425028e5b87 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:13:49.367717 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0552de-f59c-4246-8ae1-1425028e5b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:13:49.441932 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-9e0552de-f59c-4246-8ae1-1425028e5b87 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:13:49.443401 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-9e0552de-f59c-4246-8ae1-1425028e5b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:13:49.542770 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 738/1492] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Aug 31 05:13:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:13:49.825031 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe36e1d-4f5e-40b3-af97-cafe14e42936 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:13:49.826652 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fbe36e1d-4f5e-40b3-af97-cafe14e42936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:13:49.937691 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe36e1d-4f5e-40b3-af97-cafe14e42936 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:13:50.015349 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750e51db-8629-42d4-9fa9-21ce5e2cda05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:13:50.016155 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e51db-8629-42d4-9fa9-21ce5e2cda05 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:13:50.016507 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e51db-8629-42d4-9fa9-21ce5e2cda05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:13:50.016749 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e51db-8629-42d4-9fa9-21ce5e2cda05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:13:50.019128 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-750e51db-8629-42d4-9fa9-21ce5e2cda05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:13:50.032722 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:13:50.140532 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750e51db-8629-42d4-9fa9-21ce5e2cda05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:13:50.142887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750e51db-8629-42d4-9fa9-21ce5e2cda05 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:13:50.165717 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 755/1493] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:13:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 05:13:59.540712 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=82 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:14:00.163295 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=50 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:15:09.784245 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ceb65-c1b4-49fd-8fc0-e6c87a13270e None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:15:09.785177 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ceb65-c1b4-49fd-8fc0-e6c87a13270e None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:15:09.785177 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ceb65-c1b4-49fd-8fc0-e6c87a13270e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:15:09.887377 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-314ceb65-c1b4-49fd-8fc0-e6c87a13270e None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:15:09.888611 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-314ceb65-c1b4-49fd-8fc0-e6c87a13270e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:15:09.990460 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 739/1494] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:15:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:15:09.998565 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa1b70-0ecf-4476-aa37-d24b592d65d3 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:15:09.999075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa1b70-0ecf-4476-aa37-d24b592d65d3 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:15:09.999075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa1b70-0ecf-4476-aa37-d24b592d65d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:15:10.075775 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d0aa1b70-0ecf-4476-aa37-d24b592d65d3 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:15:10.076942 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d0aa1b70-0ecf-4476-aa37-d24b592d65d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:15:10.184861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 756/1495] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:15:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:15:10.191409 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314ceb65-c1b4-49fd-8fc0-e6c87a13270e None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:15:10.192636 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-314ceb65-c1b4-49fd-8fc0-e6c87a13270e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:15:10.204722 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314ceb65-c1b4-49fd-8fc0-e6c87a13270e None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:15:10.274475 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:15:10.430728 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742cd21-39c0-487c-9c38-23f9e7a7dd46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:15:10.431323 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742cd21-39c0-487c-9c38-23f9e7a7dd46 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:15:10.431596 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742cd21-39c0-487c-9c38-23f9e7a7dd46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:15:10.431861 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742cd21-39c0-487c-9c38-23f9e7a7dd46 admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:15:10.432503 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6742cd21-39c0-487c-9c38-23f9e7a7dd46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:15:10.432802 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6742cd21-39c0-487c-9c38-23f9e7a7dd46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:15:10.433808 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: WARNING py.warnings [None req-6742cd21-39c0-487c-9c38-23f9e7a7dd46 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 05:15:10.433808 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: warnings.warn(msg) Aug 31 05:15:10.433808 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: Aug 31 05:15:10.434721 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6742cd21-39c0-487c-9c38-23f9e7a7dd46 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:15:10.446550 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 740/1496] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:15:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:15:10.455160 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aa1b70-0ecf-4476-aa37-d24b592d65d3 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:15:10.464345 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aa1b70-0ecf-4476-aa37-d24b592d65d3 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:15:10.536894 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:15:10.575274 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c3184-5446-4ac9-91cb-8d6bb96870db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:15:10.575938 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c3184-5446-4ac9-91cb-8d6bb96870db admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:15:10.576208 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c3184-5446-4ac9-91cb-8d6bb96870db admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:15:10.576471 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c3184-5446-4ac9-91cb-8d6bb96870db admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:15:10.577069 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6c3184-5446-4ac9-91cb-8d6bb96870db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:15:10.578850 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: WARNING py.warnings [None req-db6c3184-5446-4ac9-91cb-8d6bb96870db admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 05:15:10.578850 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: warnings.warn(msg) Aug 31 05:15:10.578850 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: Aug 31 05:15:10.580030 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6c3184-5446-4ac9-91cb-8d6bb96870db admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:15:10.635614 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 757/1497] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:15:10 2022] POST /identity/v3/projects => generated 379 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:15:10.641174 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742cd21-39c0-487c-9c38-23f9e7a7dd46 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:15:10.641588 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-6742cd21-39c0-487c-9c38-23f9e7a7dd46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:15:10.676312 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742cd21-39c0-487c-9c38-23f9e7a7dd46 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:15:10.770422 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6876fa-1bfd-4a18-a25f-b2be4cb1d2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:15:10.771018 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6876fa-1bfd-4a18-a25f-b2be4cb1d2c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:15:10.771290 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6876fa-1bfd-4a18-a25f-b2be4cb1d2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:15:10.771578 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6876fa-1bfd-4a18-a25f-b2be4cb1d2c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:15:10.772195 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6876fa-1bfd-4a18-a25f-b2be4cb1d2c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:15:10.773207 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: WARNING py.warnings [None req-bf6876fa-1bfd-4a18-a25f-b2be4cb1d2c8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 05:15:10.773207 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: warnings.warn(msg) Aug 31 05:15:10.773207 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: Aug 31 05:15:10.774438 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6876fa-1bfd-4a18-a25f-b2be4cb1d2c8 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:15:10.817837 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 741/1498] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:15:10 2022] POST /identity/v3/users => generated 290 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:15:10.822050 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c3184-5446-4ac9-91cb-8d6bb96870db admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:15:10.823559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-db6c3184-5446-4ac9-91cb-8d6bb96870db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:15:10.887740 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c3184-5446-4ac9-91cb-8d6bb96870db admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:15:10.975716 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:15:10.988793 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61deeafb-b4b7-4496-8be3-b0e5b98e1027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:15:10.989401 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61deeafb-b4b7-4496-8be3-b0e5b98e1027 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:15:10.989682 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61deeafb-b4b7-4496-8be3-b0e5b98e1027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:15:10.989948 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61deeafb-b4b7-4496-8be3-b0e5b98e1027 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:15:10.990619 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61deeafb-b4b7-4496-8be3-b0e5b98e1027 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:15:10.991634 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: WARNING py.warnings [None req-61deeafb-b4b7-4496-8be3-b0e5b98e1027 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 05:15:10.991634 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: warnings.warn(msg) Aug 31 05:15:10.991634 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: Aug 31 05:15:10.992522 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61deeafb-b4b7-4496-8be3-b0e5b98e1027 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:15:11.006275 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 758/1499] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:15:11.012693 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6876fa-1bfd-4a18-a25f-b2be4cb1d2c8 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:15:11.014597 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-bf6876fa-1bfd-4a18-a25f-b2be4cb1d2c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:15:11.026434 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6876fa-1bfd-4a18-a25f-b2be4cb1d2c8 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:15:11.052536 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:15:11.121920 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d09e0b5-8972-41d4-bc20-b1c81964e782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:15:11.122587 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d09e0b5-8972-41d4-bc20-b1c81964e782 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:15:11.122988 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d09e0b5-8972-41d4-bc20-b1c81964e782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:15:11.123382 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d09e0b5-8972-41d4-bc20-b1c81964e782 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:15:11.124331 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d09e0b5-8972-41d4-bc20-b1c81964e782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:15:11.125911 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: WARNING py.warnings [None req-8d09e0b5-8972-41d4-bc20-b1c81964e782 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 05:15:11.125911 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: warnings.warn(msg) Aug 31 05:15:11.125911 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: Aug 31 05:15:11.127227 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d09e0b5-8972-41d4-bc20-b1c81964e782 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:15:11.144954 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 742/1500] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:15:11.151074 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61deeafb-b4b7-4496-8be3-b0e5b98e1027 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:15:11.153387 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-61deeafb-b4b7-4496-8be3-b0e5b98e1027 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:15:11.187375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61deeafb-b4b7-4496-8be3-b0e5b98e1027 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:15:11.297768 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a8fef5-7a22-4197-9ea4-356b5ab4ca5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:15:11.299061 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a8fef5-7a22-4197-9ea4-356b5ab4ca5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:15:11.299339 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a8fef5-7a22-4197-9ea4-356b5ab4ca5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:15:11.299609 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a8fef5-7a22-4197-9ea4-356b5ab4ca5d admin admin] PATH_INFO: `/v3/projects/bd5b0f82c22340829b661a18f7b001b0/users/ebb1de5b3a6f4dd4a8dc8d1fe05a53d1/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:15:11.324229 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a8fef5-7a22-4197-9ea4-356b5ab4ca5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd5b0f82c22340829b661a18f7b001b0, user_id=ebb1de5b3a6f4dd4a8dc8d1fe05a53d1, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:15:11.325267 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: WARNING py.warnings [None req-21a8fef5-7a22-4197-9ea4-356b5ab4ca5d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 05:15:11.325267 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: warnings.warn(msg) Aug 31 05:15:11.325267 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: Aug 31 05:15:11.327333 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a8fef5-7a22-4197-9ea4-356b5ab4ca5d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:15:11.355077 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 759/1501] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:15:11 2022] PUT /identity/v3/projects/bd5b0f82c22340829b661a18f7b001b0/users/ebb1de5b3a6f4dd4a8dc8d1fe05a53d1/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:15:11.368007 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad194034-a1cb-40c0-9c08-cfc8627f6d58 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:15:11.368288 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad194034-a1cb-40c0-9c08-cfc8627f6d58 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:15:11.368553 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad194034-a1cb-40c0-9c08-cfc8627f6d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:15:11.428702 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-ad194034-a1cb-40c0-9c08-cfc8627f6d58 None None] MFA Rules not processed for user `ebb1de5b3a6f4dd4a8dc8d1fe05a53d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:15:11.429828 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-ad194034-a1cb-40c0-9c08-cfc8627f6d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:15:11.524687 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 743/1502] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:15:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:15:11.538043 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0af28-bd64-4d50-9f5e-bd78512e7615 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:15:11.538367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0af28-bd64-4d50-9f5e-bd78512e7615 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:15:11.538635 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0af28-bd64-4d50-9f5e-bd78512e7615 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:15:11.539962 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 760/1503] 10.222.0.36 () {54 vars in 964 bytes} [Wed Aug 31 05:15:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:15:11.545255 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba1f1b-072d-4b71-b5ef-a1a6ea7da27f None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:15:11.545531 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba1f1b-072d-4b71-b5ef-a1a6ea7da27f None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:15:11.545784 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba1f1b-072d-4b71-b5ef-a1a6ea7da27f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:15:11.578541 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:15:11.625202 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-88ba1f1b-072d-4b71-b5ef-a1a6ea7da27f None None] MFA Rules not processed for user `a8300075371341a6b9cd4dcf8ec7fa8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:15:11.626443 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-88ba1f1b-072d-4b71-b5ef-a1a6ea7da27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:15:11.703401 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 744/1504] 10.222.0.36 () {58 vars in 1061 bytes} [Wed Aug 31 05:15:11 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:15:11.713219 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d0af28-bd64-4d50-9f5e-bd78512e7615 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:15:11.714792 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-33d0af28-bd64-4d50-9f5e-bd78512e7615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:15:11.772804 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:15:11.835619 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d0af28-bd64-4d50-9f5e-bd78512e7615 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:15:11.914736 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef69bb-e954-409e-bb09-fcb344466bb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:15:11.915336 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef69bb-e954-409e-bb09-fcb344466bb3 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:15:11.915607 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef69bb-e954-409e-bb09-fcb344466bb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:15:11.915870 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef69bb-e954-409e-bb09-fcb344466bb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:15:11.917447 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a0ef69bb-e954-409e-bb09-fcb344466bb3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:15:12.024284 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ef69bb-e954-409e-bb09-fcb344466bb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:15:12.025711 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ef69bb-e954-409e-bb09-fcb344466bb3 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:15:12.029330 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:15:12.045670 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 761/1505] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:15:11 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:15:12.098235 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a433371-a68c-4fb2-b48f-2de12df8c3f0 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:15:12.098697 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a433371-a68c-4fb2-b48f-2de12df8c3f0 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:15:12.098803 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a433371-a68c-4fb2-b48f-2de12df8c3f0 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:15:12.100059 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 745/1506] 10.222.0.36 () {52 vars in 830 bytes} [Wed Aug 31 05:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:15:12.103886 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef69bb-e954-409e-bb09-fcb344466bb3 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:15:12.105096 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a0ef69bb-e954-409e-bb09-fcb344466bb3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:15:12.207746 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef69bb-e954-409e-bb09-fcb344466bb3 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:15:12.310973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02647fb-b91e-4fbd-9eb8-fb09ba96cb4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:15:12.311590 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02647fb-b91e-4fbd-9eb8-fb09ba96cb4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:15:12.311863 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02647fb-b91e-4fbd-9eb8-fb09ba96cb4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:15:12.312134 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02647fb-b91e-4fbd-9eb8-fb09ba96cb4b admin admin] PATH_INFO: `/v3/users/ebb1de5b3a6f4dd4a8dc8d1fe05a53d1` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:15:12.325825 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02647fb-b91e-4fbd-9eb8-fb09ba96cb4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebb1de5b3a6f4dd4a8dc8d1fe05a53d1)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:15:12.326892 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: WARNING py.warnings [None req-b02647fb-b91e-4fbd-9eb8-fb09ba96cb4b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 05:15:12.326892 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: warnings.warn(msg) Aug 31 05:15:12.326892 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: Aug 31 05:15:12.328100 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02647fb-b91e-4fbd-9eb8-fb09ba96cb4b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:15:12.337882 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:15:12.399899 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b02647fb-b91e-4fbd-9eb8-fb09ba96cb4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebb1de5b3a6f4dd4a8dc8d1fe05a53d1'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:15:12.414123 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b02647fb-b91e-4fbd-9eb8-fb09ba96cb4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebb1de5b3a6f4dd4a8dc8d1fe05a53d1'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:15:12.425820 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b02647fb-b91e-4fbd-9eb8-fb09ba96cb4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebb1de5b3a6f4dd4a8dc8d1fe05a53d1'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:15:12.426353 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b02647fb-b91e-4fbd-9eb8-fb09ba96cb4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebb1de5b3a6f4dd4a8dc8d1fe05a53d1'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:15:12.454316 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b02647fb-b91e-4fbd-9eb8-fb09ba96cb4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebb1de5b3a6f4dd4a8dc8d1fe05a53d1'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:15:12.460694 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 762/1507] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:15:12 2022] DELETE /identity/v3/users/ebb1de5b3a6f4dd4a8dc8d1fe05a53d1 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:15:12.465465 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576cc2c9-82c1-46a3-a296-24715e75b82b None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:15:12.465750 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576cc2c9-82c1-46a3-a296-24715e75b82b None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:15:12.466051 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576cc2c9-82c1-46a3-a296-24715e75b82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:15:12.524433 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-576cc2c9-82c1-46a3-a296-24715e75b82b None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:15:12.525646 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-576cc2c9-82c1-46a3-a296-24715e75b82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:15:12.604375 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:15:12.617544 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 746/1508] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:15:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:15:12.629045 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02647fb-b91e-4fbd-9eb8-fb09ba96cb4b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:15:12.630302 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b02647fb-b91e-4fbd-9eb8-fb09ba96cb4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:15:12.735155 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02647fb-b91e-4fbd-9eb8-fb09ba96cb4b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:15:12.792095 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:15:12.817125 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c34f579-a8c9-4377-8044-04341cd0890d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:15:12.818055 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34f579-a8c9-4377-8044-04341cd0890d service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:15:12.818466 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34f579-a8c9-4377-8044-04341cd0890d service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:15:12.818850 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34f579-a8c9-4377-8044-04341cd0890d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:15:12.821449 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1c34f579-a8c9-4377-8044-04341cd0890d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:15:12.941260 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c34f579-a8c9-4377-8044-04341cd0890d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:15:12.942671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c34f579-a8c9-4377-8044-04341cd0890d service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:15:12.963606 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 763/1509] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:15:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:15:13.326802 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576cc2c9-82c1-46a3-a296-24715e75b82b None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:15:13.328163 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-576cc2c9-82c1-46a3-a296-24715e75b82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:15:13.340338 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576cc2c9-82c1-46a3-a296-24715e75b82b None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:15:13.437384 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6664bf4-3bc7-4e33-8887-02b58010ae52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:15:13.438030 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6664bf4-3bc7-4e33-8887-02b58010ae52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:15:13.438319 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6664bf4-3bc7-4e33-8887-02b58010ae52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:15:13.438585 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6664bf4-3bc7-4e33-8887-02b58010ae52 admin admin] PATH_INFO: `/v3/projects/bd5b0f82c22340829b661a18f7b001b0` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:15:13.452908 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6664bf4-3bc7-4e33-8887-02b58010ae52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd5b0f82c22340829b661a18f7b001b0)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:15:13.454419 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: WARNING py.warnings [None req-a6664bf4-3bc7-4e33-8887-02b58010ae52 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 05:15:13.454419 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: warnings.warn(msg) Aug 31 05:15:13.454419 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: Aug 31 05:15:13.456174 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6664bf4-3bc7-4e33-8887-02b58010ae52 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:15:13.521530 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a6664bf4-3bc7-4e33-8887-02b58010ae52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd5b0f82c22340829b661a18f7b001b0', 'request_id': 'req-a6664bf4-3bc7-4e33-8887-02b58010ae52'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:15:13.534732 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a6664bf4-3bc7-4e33-8887-02b58010ae52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd5b0f82c22340829b661a18f7b001b0', 'request_id': 'req-a6664bf4-3bc7-4e33-8887-02b58010ae52'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:15:13.541176 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a6664bf4-3bc7-4e33-8887-02b58010ae52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd5b0f82c22340829b661a18f7b001b0', 'request_id': 'req-a6664bf4-3bc7-4e33-8887-02b58010ae52'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:15:13.569597 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a6664bf4-3bc7-4e33-8887-02b58010ae52 admin admin] The token cache is being invalidate because project bd5b0f82c22340829b661a18f7b001b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:15:13.569876 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a6664bf4-3bc7-4e33-8887-02b58010ae52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:15:13.570397 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a6664bf4-3bc7-4e33-8887-02b58010ae52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:15:13.571738 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 747/1510] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:15:13 2022] DELETE /identity/v3/projects/bd5b0f82c22340829b661a18f7b001b0 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:15:22.961297 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=62 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:15:23.567654 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:16:38.927746 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56a77f-59df-43d3-a2fa-ebaeddcfe78f None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:16:38.928331 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56a77f-59df-43d3-a2fa-ebaeddcfe78f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:16:38.928331 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56a77f-59df-43d3-a2fa-ebaeddcfe78f None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:16:38.929848 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 764/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 05:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:16:38.935702 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8bc93b-ab52-499b-9e5b-14ef16891e86 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:16:38.936456 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8bc93b-ab52-499b-9e5b-14ef16891e86 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:16:38.936615 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8bc93b-ab52-499b-9e5b-14ef16891e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:16:39.036951 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-ba8bc93b-ab52-499b-9e5b-14ef16891e86 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:16:39.038463 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-ba8bc93b-ab52-499b-9e5b-14ef16891e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:16:39.149946 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 748/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 05:16:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:16:39.305377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d56a77f-59df-43d3-a2fa-ebaeddcfe78f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:16:39.309329 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6d56a77f-59df-43d3-a2fa-ebaeddcfe78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:16:39.448789 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d56a77f-59df-43d3-a2fa-ebaeddcfe78f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:16:39.454791 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97551895-98c6-4295-a5ff-29304a876e25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:16:39.455424 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97551895-98c6-4295-a5ff-29304a876e25 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:16:39.455653 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97551895-98c6-4295-a5ff-29304a876e25 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:16:39.455941 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97551895-98c6-4295-a5ff-29304a876e25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:16:39.464310 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97551895-98c6-4295-a5ff-29304a876e25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:16:39.465898 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97551895-98c6-4295-a5ff-29304a876e25 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:16:39.475283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 765/1513] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:16:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:16:40.299799 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7af7b-2b4b-4ea7-aee7-d6bf965e5993 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:16:40.300366 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7af7b-2b4b-4ea7-aee7-d6bf965e5993 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:16:40.300366 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7af7b-2b4b-4ea7-aee7-d6bf965e5993 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:16:40.301715 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 749/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 05:16:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:16:40.307716 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f06f65-7ae6-4651-86e9-a2dcff282540 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:16:40.308184 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f06f65-7ae6-4651-86e9-a2dcff282540 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:16:40.308295 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f06f65-7ae6-4651-86e9-a2dcff282540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:16:40.333279 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a6f06f65-7ae6-4651-86e9-a2dcff282540 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:16:40.334501 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a6f06f65-7ae6-4651-86e9-a2dcff282540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:16:40.343121 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 766/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 05:16:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:16:40.501160 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7af7b-2b4b-4ea7-aee7-d6bf965e5993 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:16:40.524535 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7af7b-2b4b-4ea7-aee7-d6bf965e5993 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:16:40.531156 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3bb381-810a-4d66-8dcd-c8735ad3af10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:16:40.531760 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3bb381-810a-4d66-8dcd-c8735ad3af10 service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:16:40.532032 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3bb381-810a-4d66-8dcd-c8735ad3af10 service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:16:40.532326 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3bb381-810a-4d66-8dcd-c8735ad3af10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:16:40.541144 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3bb381-810a-4d66-8dcd-c8735ad3af10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:16:40.542774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3bb381-810a-4d66-8dcd-c8735ad3af10 service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:16:40.552683 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 750/1516] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:16:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:16:41.299951 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f60dd33-45ea-4890-82ca-cb15b0c7f991 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:16:41.300366 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f60dd33-45ea-4890-82ca-cb15b0c7f991 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:16:41.300812 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f60dd33-45ea-4890-82ca-cb15b0c7f991 None None] PATH_INFO: `/v3` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:16:41.302611 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 767/1517] 10.222.0.43 () {54 vars in 925 bytes} [Wed Aug 31 05:16:41 2022] GET /identity/v3 => generated 255 bytes in 5 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:16:41.317291 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94aceb-514d-4571-9891-a527a490a0e9 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:16:41.317818 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94aceb-514d-4571-9891-a527a490a0e9 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:16:41.317818 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94aceb-514d-4571-9891-a527a490a0e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:16:41.392002 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-0a94aceb-514d-4571-9891-a527a490a0e9 None None] MFA Rules not processed for user `bd4387035099416789009f0ecb8ad282`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:16:41.393226 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-0a94aceb-514d-4571-9891-a527a490a0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:16:41.412618 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:16:41.471131 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 751/1518] 10.222.0.43 () {58 vars in 1015 bytes} [Wed Aug 31 05:16:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:16:50.327878 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=56 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:16:50.820091 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5301be0-0549-4451-abf1-6d06c0a11538 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:16:50.820328 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5301be0-0549-4451-abf1-6d06c0a11538 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:16:50.820531 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5301be0-0549-4451-abf1-6d06c0a11538 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:16:50.821919 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 768/1519] 10.222.0.36 () {54 vars in 961 bytes} [Wed Aug 31 05:16:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:16:50.827199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b35c0-4379-47c6-baf2-477eb153f329 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:16:50.827723 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b35c0-4379-47c6-baf2-477eb153f329 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:16:50.827723 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b35c0-4379-47c6-baf2-477eb153f329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:16:50.848066 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-fc5b35c0-4379-47c6-baf2-477eb153f329 None None] MFA Rules not processed for user `bd4387035099416789009f0ecb8ad282`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:16:50.849254 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-fc5b35c0-4379-47c6-baf2-477eb153f329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:16:50.858478 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 752/1520] 10.222.0.36 () {58 vars in 1058 bytes} [Wed Aug 31 05:16:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:16:50.866718 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5301be0-0549-4451-abf1-6d06c0a11538 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:16:50.880678 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5301be0-0549-4451-abf1-6d06c0a11538 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:16:50.885883 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564310cc-7ae8-4b88-b9c7-128f71797f9a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd4387035099416789009f0ecb8ad282', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:16:50.886485 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564310cc-7ae8-4b88-b9c7-128f71797f9a service placement] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:16:50.886767 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564310cc-7ae8-4b88-b9c7-128f71797f9a service placement] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:16:50.887054 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564310cc-7ae8-4b88-b9c7-128f71797f9a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:16:50.895646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564310cc-7ae8-4b88-b9c7-128f71797f9a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:16:50.897250 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564310cc-7ae8-4b88-b9c7-128f71797f9a service placement] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:16:50.907169 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 769/1521] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:16:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:16:51.786603 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8041cdf-fed2-441a-a349-0f741ee25fa3 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:16:51.786975 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8041cdf-fed2-441a-a349-0f741ee25fa3 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:16:51.787324 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8041cdf-fed2-441a-a349-0f741ee25fa3 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:16:51.790115 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 753/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 05:16:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:16:51.796396 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81cde1-dff3-4af0-bee8-3c12567d1fd9 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:16:51.797267 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81cde1-dff3-4af0-bee8-3c12567d1fd9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:16:51.797267 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81cde1-dff3-4af0-bee8-3c12567d1fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:16:51.820242 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ba81cde1-dff3-4af0-bee8-3c12567d1fd9 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:16:51.821430 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ba81cde1-dff3-4af0-bee8-3c12567d1fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:16:51.829729 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 770/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 05:16:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:16:51.983429 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8041cdf-fed2-441a-a349-0f741ee25fa3 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:16:52.001372 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8041cdf-fed2-441a-a349-0f741ee25fa3 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:16:52.007009 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a466e-6aac-44d0-9f19-09ebf1171ed2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:16:52.007603 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a466e-6aac-44d0-9f19-09ebf1171ed2 service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:16:52.007875 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a466e-6aac-44d0-9f19-09ebf1171ed2 service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:16:52.008143 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a466e-6aac-44d0-9f19-09ebf1171ed2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:16:52.016291 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055a466e-6aac-44d0-9f19-09ebf1171ed2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:16:52.017895 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055a466e-6aac-44d0-9f19-09ebf1171ed2 service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:16:52.027227 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 754/1524] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:16:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:16:52.840794 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74fc09-7a1a-4b82-bee2-1ca120be96c2 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:16:52.841416 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74fc09-7a1a-4b82-bee2-1ca120be96c2 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:16:52.841416 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74fc09-7a1a-4b82-bee2-1ca120be96c2 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:16:52.842752 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 771/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 05:16:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:16:52.848758 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef53b6-3b47-4d6c-be7e-77ac506c7c4e None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:16:52.849231 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef53b6-3b47-4d6c-be7e-77ac506c7c4e None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:16:52.849710 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef53b6-3b47-4d6c-be7e-77ac506c7c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:16:52.873557 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-a6ef53b6-3b47-4d6c-be7e-77ac506c7c4e None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:16:52.874807 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a6ef53b6-3b47-4d6c-be7e-77ac506c7c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:16:52.888398 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 755/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 05:16:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:16:53.041941 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f74fc09-7a1a-4b82-bee2-1ca120be96c2 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:16:53.058196 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f74fc09-7a1a-4b82-bee2-1ca120be96c2 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:16:53.064706 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55716f6-3bb7-4681-a1a8-56593fb73530 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:16:53.065303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55716f6-3bb7-4681-a1a8-56593fb73530 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:16:53.065582 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55716f6-3bb7-4681-a1a8-56593fb73530 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:16:53.065844 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55716f6-3bb7-4681-a1a8-56593fb73530 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:16:53.074169 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55716f6-3bb7-4681-a1a8-56593fb73530 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:16:53.075743 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55716f6-3bb7-4681-a1a8-56593fb73530 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:16:53.085205 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 772/1527] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:16:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:16:56.745284 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24aa0d1-9b53-4064-8522-6cef676708b6 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:16:56.745926 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24aa0d1-9b53-4064-8522-6cef676708b6 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:16:56.745926 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24aa0d1-9b53-4064-8522-6cef676708b6 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:16:56.748246 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 756/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 05:16:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:16:56.755335 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169af4a6-8e09-4cb5-899b-2ac687d0e0e7 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:16:56.755877 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169af4a6-8e09-4cb5-899b-2ac687d0e0e7 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:16:56.756154 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169af4a6-8e09-4cb5-899b-2ac687d0e0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:16:56.776408 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-169af4a6-8e09-4cb5-899b-2ac687d0e0e7 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:16:56.777616 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-169af4a6-8e09-4cb5-899b-2ac687d0e0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:16:56.785796 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 773/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 05:16:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:16:57.083805 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24aa0d1-9b53-4064-8522-6cef676708b6 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:16:57.086679 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-e24aa0d1-9b53-4064-8522-6cef676708b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:16:57.172310 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24aa0d1-9b53-4064-8522-6cef676708b6 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:16:57.176390 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcdedc3-937d-484b-a90d-be00a16e8f6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:16:57.177032 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdedc3-937d-484b-a90d-be00a16e8f6d service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:16:57.177308 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdedc3-937d-484b-a90d-be00a16e8f6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:16:57.177581 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdedc3-937d-484b-a90d-be00a16e8f6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:16:57.187514 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcdedc3-937d-484b-a90d-be00a16e8f6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:16:57.188954 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcdedc3-937d-484b-a90d-be00a16e8f6d service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:16:57.198656 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 757/1530] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:16:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:16:59.089491 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580416e9-0b41-4076-8617-306e16721db7 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:16:59.090121 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580416e9-0b41-4076-8617-306e16721db7 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:16:59.090121 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580416e9-0b41-4076-8617-306e16721db7 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:16:59.091432 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 774/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 05:16:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:16:59.097761 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a2678-cea5-401b-adff-021760d12782 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:16:59.098503 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a2678-cea5-401b-adff-021760d12782 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:16:59.098604 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a2678-cea5-401b-adff-021760d12782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:16:59.130591 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-632a2678-cea5-401b-adff-021760d12782 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:16:59.131804 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-632a2678-cea5-401b-adff-021760d12782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:16:59.141065 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 758/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 05:16:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:16:59.436741 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580416e9-0b41-4076-8617-306e16721db7 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:16:59.439681 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-580416e9-0b41-4076-8617-306e16721db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:16:59.456922 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580416e9-0b41-4076-8617-306e16721db7 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:16:59.460997 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a5a2f-8971-4baf-bbe1-a076511fcc22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:16:59.461601 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a5a2f-8971-4baf-bbe1-a076511fcc22 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:16:59.461872 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a5a2f-8971-4baf-bbe1-a076511fcc22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:16:59.462182 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a5a2f-8971-4baf-bbe1-a076511fcc22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:16:59.471943 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05a5a2f-8971-4baf-bbe1-a076511fcc22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:16:59.473375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05a5a2f-8971-4baf-bbe1-a076511fcc22 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:16:59.483152 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 775/1533] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:16:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:17:02.759751 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f509217-2a3a-43cf-9ee6-8597e881549f None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:17:02.760623 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f509217-2a3a-43cf-9ee6-8597e881549f None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:17:02.760623 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f509217-2a3a-43cf-9ee6-8597e881549f None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:17:02.762712 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 759/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 05:17:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:17:02.771474 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6bbdf9-5b0e-48a4-b333-b703912dddcc None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:17:02.771905 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6bbdf9-5b0e-48a4-b333-b703912dddcc None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:17:02.772050 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6bbdf9-5b0e-48a4-b333-b703912dddcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:17:02.793399 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-bb6bbdf9-5b0e-48a4-b333-b703912dddcc None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:17:02.794662 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-bb6bbdf9-5b0e-48a4-b333-b703912dddcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:17:02.803608 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 776/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 05:17:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:17:03.354335 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f509217-2a3a-43cf-9ee6-8597e881549f None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:17:03.356420 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-9f509217-2a3a-43cf-9ee6-8597e881549f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:17:03.435945 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:17:03.455426 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f509217-2a3a-43cf-9ee6-8597e881549f None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:17:03.461831 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e8f7d5-bf2b-45e8-978f-1bba476a2ee0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:17:03.462557 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8f7d5-bf2b-45e8-978f-1bba476a2ee0 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:17:03.462836 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8f7d5-bf2b-45e8-978f-1bba476a2ee0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:17:03.463109 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8f7d5-bf2b-45e8-978f-1bba476a2ee0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:17:03.473390 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e8f7d5-bf2b-45e8-978f-1bba476a2ee0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:17:03.474904 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e8f7d5-bf2b-45e8-978f-1bba476a2ee0 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:17:03.484748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 760/1536] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:17:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:17:05.414438 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0754733-72af-48b1-98a7-af6858ea6c7b None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:17:05.415055 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0754733-72af-48b1-98a7-af6858ea6c7b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:17:05.415055 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0754733-72af-48b1-98a7-af6858ea6c7b None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:17:05.416535 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 777/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 05:17:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:17:05.422175 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be422bbb-03ea-4c0a-aa4f-243eccda08e6 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:17:05.422649 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be422bbb-03ea-4c0a-aa4f-243eccda08e6 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:17:05.422775 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be422bbb-03ea-4c0a-aa4f-243eccda08e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:17:05.444355 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-be422bbb-03ea-4c0a-aa4f-243eccda08e6 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:17:05.445555 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-be422bbb-03ea-4c0a-aa4f-243eccda08e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:17:05.454260 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 761/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 05:17:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:17:06.019266 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0754733-72af-48b1-98a7-af6858ea6c7b None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:17:06.021146 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a0754733-72af-48b1-98a7-af6858ea6c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:17:06.035785 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0754733-72af-48b1-98a7-af6858ea6c7b None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:17:06.039923 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6fe49b-2b6a-4642-94ad-4c6d55a80535 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:17:06.040594 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6fe49b-2b6a-4642-94ad-4c6d55a80535 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:17:06.040873 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6fe49b-2b6a-4642-94ad-4c6d55a80535 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:17:06.041144 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6fe49b-2b6a-4642-94ad-4c6d55a80535 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:17:06.051310 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6fe49b-2b6a-4642-94ad-4c6d55a80535 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:17:06.052629 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6fe49b-2b6a-4642-94ad-4c6d55a80535 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:17:06.063392 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 778/1539] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:17:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:17:15.439134 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=11 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:17:16.059027 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=33 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:29.284151 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc3f39-663f-49ea-bc45-cfd08f7b7076 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:29.285327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc3f39-663f-49ea-bc45-cfd08f7b7076 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:29.285854 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc3f39-663f-49ea-bc45-cfd08f7b7076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:29.310573 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-bddc3f39-663f-49ea-bc45-cfd08f7b7076 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:29.311871 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-bddc3f39-663f-49ea-bc45-cfd08f7b7076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:29.325429 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 762/1540] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:29.334854 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e19df-454f-41b6-bec5-9cd9122804d7 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:29.335981 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e19df-454f-41b6-bec5-9cd9122804d7 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:29.335981 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e19df-454f-41b6-bec5-9cd9122804d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:29.364385 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b90e19df-454f-41b6-bec5-9cd9122804d7 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:29.365618 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b90e19df-454f-41b6-bec5-9cd9122804d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:29.376935 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 779/1541] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:29.381379 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc3f39-663f-49ea-bc45-cfd08f7b7076 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:29.395379 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc3f39-663f-49ea-bc45-cfd08f7b7076 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:29.404189 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30fdd79-fc55-4ffe-8c89-d2d1c3a87af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:29.404816 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30fdd79-fc55-4ffe-8c89-d2d1c3a87af2 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:29.405094 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30fdd79-fc55-4ffe-8c89-d2d1c3a87af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:29.405365 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30fdd79-fc55-4ffe-8c89-d2d1c3a87af2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:29.406056 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30fdd79-fc55-4ffe-8c89-d2d1c3a87af2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:18:29.406349 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30fdd79-fc55-4ffe-8c89-d2d1c3a87af2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:29.408047 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30fdd79-fc55-4ffe-8c89-d2d1c3a87af2 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:29.425359 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 763/1542] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:18:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:18:29.433238 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90e19df-454f-41b6-bec5-9cd9122804d7 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:29.439206 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a2348-77d2-4e92-bcbf-7b6edb948b0a None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:29.439543 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a2348-77d2-4e92-bcbf-7b6edb948b0a None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:29.439761 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a2348-77d2-4e92-bcbf-7b6edb948b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:29.443625 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90e19df-454f-41b6-bec5-9cd9122804d7 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:29.454197 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36100b-5c49-4107-a648-51b30e749768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:29.454830 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36100b-5c49-4107-a648-51b30e749768 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:29.455102 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36100b-5c49-4107-a648-51b30e749768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:29.455364 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36100b-5c49-4107-a648-51b30e749768 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:29.456016 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a36100b-5c49-4107-a648-51b30e749768 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:29.458385 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a36100b-5c49-4107-a648-51b30e749768 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:29.479245 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-e36a2348-77d2-4e92-bcbf-7b6edb948b0a None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:29.480515 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-e36a2348-77d2-4e92-bcbf-7b6edb948b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:29.511620 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 764/1543] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:29.512992 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6439ad-e513-4453-ad36-2b19e86ace41 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:29.513280 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6439ad-e513-4453-ad36-2b19e86ace41 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:29.513571 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6439ad-e513-4453-ad36-2b19e86ace41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:29.536668 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 780/1544] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:18:29 2022] POST /identity/v3/projects => generated 379 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:18:29.538716 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec034c6-4901-4564-a24e-41113db4050f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:29.539131 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec034c6-4901-4564-a24e-41113db4050f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:29.539402 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec034c6-4901-4564-a24e-41113db4050f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:29.570334 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-7f6439ad-e513-4453-ad36-2b19e86ace41 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:29.571593 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-7f6439ad-e513-4453-ad36-2b19e86ace41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:29.575566 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-aec034c6-4901-4564-a24e-41113db4050f None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:29.576834 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-aec034c6-4901-4564-a24e-41113db4050f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:29.661986 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 781/1545] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:29.663472 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec034c6-4901-4564-a24e-41113db4050f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:29.675499 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec034c6-4901-4564-a24e-41113db4050f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:29.680842 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1234539f-4443-45f3-b894-e11257b68a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:29.681844 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234539f-4443-45f3-b894-e11257b68a8a admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:29.682359 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234539f-4443-45f3-b894-e11257b68a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:29.682819 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234539f-4443-45f3-b894-e11257b68a8a admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:29.684175 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1234539f-4443-45f3-b894-e11257b68a8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:29.685409 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: WARNING py.warnings [None req-1234539f-4443-45f3-b894-e11257b68a8a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 05:18:29.685409 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: warnings.warn(msg) Aug 31 05:18:29.685409 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: Aug 31 05:18:29.687190 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1234539f-4443-45f3-b894-e11257b68a8a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:29.714332 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 765/1546] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:29.715694 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f83b03-ec2e-4510-9e89-83da026f8dbf None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:29.716024 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f83b03-ec2e-4510-9e89-83da026f8dbf None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:29.716344 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f83b03-ec2e-4510-9e89-83da026f8dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:29.719221 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 782/1547] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:18:29 2022] POST /identity/v3/users => generated 290 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:18:29.723414 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1234539f-4443-45f3-b894-e11257b68a8a admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:29.739975 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1234539f-4443-45f3-b894-e11257b68a8a admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:29.749669 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8ab106-7737-44c7-b628-8c194062d149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:29.750751 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8ab106-7737-44c7-b628-8c194062d149 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:29.751288 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8ab106-7737-44c7-b628-8c194062d149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:29.751763 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8ab106-7737-44c7-b628-8c194062d149 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:29.752632 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8ab106-7737-44c7-b628-8c194062d149 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:18:29.753154 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8ab106-7737-44c7-b628-8c194062d149 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:29.757814 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: WARNING py.warnings [None req-2c8ab106-7737-44c7-b628-8c194062d149 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 05:18:29.757814 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: warnings.warn(msg) Aug 31 05:18:29.757814 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: Aug 31 05:18:29.757814 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8ab106-7737-44c7-b628-8c194062d149 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:29.771075 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-b5f83b03-ec2e-4510-9e89-83da026f8dbf None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:29.772310 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b5f83b03-ec2e-4510-9e89-83da026f8dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:29.779602 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 783/1548] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:18:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:18:29.780537 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8ab106-7737-44c7-b628-8c194062d149 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:29.785632 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 766/1549] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:29.787399 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e82368e-97a4-4148-96f1-a4c25c518971 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:29.787921 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e82368e-97a4-4148-96f1-a4c25c518971 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:29.788408 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e82368e-97a4-4148-96f1-a4c25c518971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:29.800198 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8ab106-7737-44c7-b628-8c194062d149 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:29.804796 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85a4e79-82ce-4451-afa4-83bb24ef6c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:29.805408 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a4e79-82ce-4451-afa4-83bb24ef6c55 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:29.805693 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a4e79-82ce-4451-afa4-83bb24ef6c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:29.805941 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a4e79-82ce-4451-afa4-83bb24ef6c55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:29.806598 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85a4e79-82ce-4451-afa4-83bb24ef6c55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:29.808189 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85a4e79-82ce-4451-afa4-83bb24ef6c55 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:29.809911 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-5e82368e-97a4-4148-96f1-a4c25c518971 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:29.811126 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-5e82368e-97a4-4148-96f1-a4c25c518971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:29.819055 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 784/1550] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:18:29.820183 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c709ef-820c-44e1-aa4d-3bc264dd50b5 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:29.820461 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c709ef-820c-44e1-aa4d-3bc264dd50b5 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:29.820728 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c709ef-820c-44e1-aa4d-3bc264dd50b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:29.822722 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 767/1551] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:29.823447 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e82368e-97a4-4148-96f1-a4c25c518971 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:29.834055 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e82368e-97a4-4148-96f1-a4c25c518971 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:29.838572 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e847cb-026f-4e91-8bea-cfd9a5aff7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:29.839165 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e847cb-026f-4e91-8bea-cfd9a5aff7d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:29.839436 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e847cb-026f-4e91-8bea-cfd9a5aff7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:29.839697 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e847cb-026f-4e91-8bea-cfd9a5aff7d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:29.840324 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e847cb-026f-4e91-8bea-cfd9a5aff7d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:29.841340 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: WARNING py.warnings [None req-81e847cb-026f-4e91-8bea-cfd9a5aff7d7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 05:18:29.841340 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: warnings.warn(msg) Aug 31 05:18:29.841340 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: Aug 31 05:18:29.842219 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-42c709ef-820c-44e1-aa4d-3bc264dd50b5 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:29.842606 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e847cb-026f-4e91-8bea-cfd9a5aff7d7 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:29.843413 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-42c709ef-820c-44e1-aa4d-3bc264dd50b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:29.854508 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 785/1552] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:29.854508 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b6f53-4464-4329-9e99-5647318fa8c0 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:29.854508 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b6f53-4464-4329-9e99-5647318fa8c0 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:29.854508 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b6f53-4464-4329-9e99-5647318fa8c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:29.876438 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 768/1553] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:18:29 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:18:29.877262 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-973b6f53-4464-4329-9e99-5647318fa8c0 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:29.877541 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0537e456-74bd-4f65-b3c2-b84089b99bf6 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:29.877829 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0537e456-74bd-4f65-b3c2-b84089b99bf6 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:29.878151 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0537e456-74bd-4f65-b3c2-b84089b99bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:29.878499 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-973b6f53-4464-4329-9e99-5647318fa8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:29.898295 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-0537e456-74bd-4f65-b3c2-b84089b99bf6 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:29.899503 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-0537e456-74bd-4f65-b3c2-b84089b99bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:29.936837 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 786/1554] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:29.938252 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036fc19-2c54-489c-abf2-ef7de3e411db None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:29.938567 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036fc19-2c54-489c-abf2-ef7de3e411db None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:29.938835 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036fc19-2c54-489c-abf2-ef7de3e411db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:29.959817 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-8036fc19-2c54-489c-abf2-ef7de3e411db None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:29.960045 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 769/1555] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:29.960819 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0537e456-74bd-4f65-b3c2-b84089b99bf6 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:29.961080 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8036fc19-2c54-489c-abf2-ef7de3e411db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:29.970123 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0537e456-74bd-4f65-b3c2-b84089b99bf6 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:29.977341 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 787/1556] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:29.977341 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8036fc19-2c54-489c-abf2-ef7de3e411db None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:29.978260 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e2dd8-60c1-4bb8-a727-751d44fd5b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:29.978870 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e2dd8-60c1-4bb8-a727-751d44fd5b39 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:29.979222 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e2dd8-60c1-4bb8-a727-751d44fd5b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:29.979409 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e2dd8-60c1-4bb8-a727-751d44fd5b39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:29.980061 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66e2dd8-60c1-4bb8-a727-751d44fd5b39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:29.981680 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66e2dd8-60c1-4bb8-a727-751d44fd5b39 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:29.984149 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8036fc19-2c54-489c-abf2-ef7de3e411db None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:29.989042 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f541b8-97a7-4335-b60a-32d0d42aad73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:29.989630 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f541b8-97a7-4335-b60a-32d0d42aad73 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:29.989921 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f541b8-97a7-4335-b60a-32d0d42aad73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:29.990238 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f541b8-97a7-4335-b60a-32d0d42aad73 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:29.990875 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f541b8-97a7-4335-b60a-32d0d42aad73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:18:29.991171 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f541b8-97a7-4335-b60a-32d0d42aad73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:29.992325 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 770/1557] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:18:29.992739 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f541b8-97a7-4335-b60a-32d0d42aad73 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:29.993601 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc82658-0ce6-4db4-a5ba-084635955f44 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:29.993884 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc82658-0ce6-4db4-a5ba-084635955f44 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:29.994277 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc82658-0ce6-4db4-a5ba-084635955f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:29.996187 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:30.007875 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 788/1558] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:18:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:18:30.009182 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eaaa17-f752-449a-b389-c936815808ee None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:30.009461 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eaaa17-f752-449a-b389-c936815808ee None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:30.009726 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eaaa17-f752-449a-b389-c936815808ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:30.014228 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-cbc82658-0ce6-4db4-a5ba-084635955f44 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:30.015514 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-cbc82658-0ce6-4db4-a5ba-084635955f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:30.027076 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 771/1559] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:30.027076 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc82658-0ce6-4db4-a5ba-084635955f44 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:30.029013 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-55eaaa17-f752-449a-b389-c936815808ee None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:30.030215 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-55eaaa17-f752-449a-b389-c936815808ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:30.033398 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc82658-0ce6-4db4-a5ba-084635955f44 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:30.037908 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6097db18-208b-4f4c-9350-f90cddb89fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:30.038078 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 789/1560] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:30.039375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f63dd-5901-4e98-9bc0-ec6480d616da None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:30.039499 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f63dd-5901-4e98-9bc0-ec6480d616da None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:30.039732 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f63dd-5901-4e98-9bc0-ec6480d616da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:30.045718 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097db18-208b-4f4c-9350-f90cddb89fc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:30.046099 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097db18-208b-4f4c-9350-f90cddb89fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:30.046375 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097db18-208b-4f4c-9350-f90cddb89fc5 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:30.047027 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6097db18-208b-4f4c-9350-f90cddb89fc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:30.049072 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6097db18-208b-4f4c-9350-f90cddb89fc5 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:30.058404 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-001f63dd-5901-4e98-9bc0-ec6480d616da None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:30.059626 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-001f63dd-5901-4e98-9bc0-ec6480d616da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:30.068602 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 790/1561] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:30.069639 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a8617-5a1a-4e76-a8d1-474cbc8cb736 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:30.069916 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a8617-5a1a-4e76-a8d1-474cbc8cb736 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:30.070210 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a8617-5a1a-4e76-a8d1-474cbc8cb736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:30.074445 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 772/1562] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:18:30 2022] POST /identity/v3/users => generated 293 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:18:30.074445 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6097db18-208b-4f4c-9350-f90cddb89fc5 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:30.079929 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:30.084014 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6097db18-208b-4f4c-9350-f90cddb89fc5 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:30.089503 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-857a8617-5a1a-4e76-a8d1-474cbc8cb736 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:30.089730 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a2050e-2c0c-414f-a9a8-16bcefd3556d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:30.090336 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2050e-2c0c-414f-a9a8-16bcefd3556d admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:30.090610 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2050e-2c0c-414f-a9a8-16bcefd3556d admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:30.090772 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-857a8617-5a1a-4e76-a8d1-474cbc8cb736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:30.090884 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2050e-2c0c-414f-a9a8-16bcefd3556d admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:30.091529 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a2050e-2c0c-414f-a9a8-16bcefd3556d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:18:30.091823 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a2050e-2c0c-414f-a9a8-16bcefd3556d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:30.093449 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a2050e-2c0c-414f-a9a8-16bcefd3556d admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:30.098824 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 791/1563] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:30.099338 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857a8617-5a1a-4e76-a8d1-474cbc8cb736 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:30.113256 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 773/1564] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:18:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:18:30.113256 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a2050e-2c0c-414f-a9a8-16bcefd3556d admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:30.113549 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857a8617-5a1a-4e76-a8d1-474cbc8cb736 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:30.117524 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23272d4-3095-4993-b610-8cd2140779d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:30.118287 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23272d4-3095-4993-b610-8cd2140779d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:30.118630 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23272d4-3095-4993-b610-8cd2140779d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:30.118958 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23272d4-3095-4993-b610-8cd2140779d8 admin admin] PATH_INFO: `/v3/projects/44a0d4c08d8044b1ab696dc757ba1a4d/users/69e6e2487fce4c03a8770443791c5b8a/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:30.119495 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a2050e-2c0c-414f-a9a8-16bcefd3556d admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:30.125231 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392becc-9526-4542-b308-e56777c5216e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:30.125857 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392becc-9526-4542-b308-e56777c5216e admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:30.126228 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392becc-9526-4542-b308-e56777c5216e admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:30.126503 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392becc-9526-4542-b308-e56777c5216e admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:30.127136 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b392becc-9526-4542-b308-e56777c5216e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:30.129532 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b392becc-9526-4542-b308-e56777c5216e admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:30.133663 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23272d4-3095-4993-b610-8cd2140779d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44a0d4c08d8044b1ab696dc757ba1a4d, user_id=69e6e2487fce4c03a8770443791c5b8a, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:30.136868 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23272d4-3095-4993-b610-8cd2140779d8 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:30.151043 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 792/1565] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:18:30 2022] PUT /identity/v3/projects/44a0d4c08d8044b1ab696dc757ba1a4d/users/69e6e2487fce4c03a8770443791c5b8a/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:30.152035 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23272d4-3095-4993-b610-8cd2140779d8 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:30.157863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 774/1566] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:18:30 2022] POST /identity/v3/projects => generated 403 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:18:30.161242 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec7fce-98d5-44aa-a1a6-46bdf82779b1 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:30.161242 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec7fce-98d5-44aa-a1a6-46bdf82779b1 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:30.161242 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec7fce-98d5-44aa-a1a6-46bdf82779b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:30.179573 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-24ec7fce-98d5-44aa-a1a6-46bdf82779b1 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:30.180773 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-24ec7fce-98d5-44aa-a1a6-46bdf82779b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:30.220106 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23272d4-3095-4993-b610-8cd2140779d8 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:30.246070 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 775/1567] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:30.246927 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec7fce-98d5-44aa-a1a6-46bdf82779b1 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:30.257019 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec7fce-98d5-44aa-a1a6-46bdf82779b1 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:30.261666 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ed17a-b381-4542-8e9a-beee5d141025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:30.262259 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ed17a-b381-4542-8e9a-beee5d141025 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:30.262541 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ed17a-b381-4542-8e9a-beee5d141025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:30.262810 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ed17a-b381-4542-8e9a-beee5d141025 admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:30.263456 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ed17a-b381-4542-8e9a-beee5d141025 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:18:30.263758 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ed17a-b381-4542-8e9a-beee5d141025 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:30.265393 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ed17a-b381-4542-8e9a-beee5d141025 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:30.277291 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 776/1568] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:18:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:18:30.278121 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ed17a-b381-4542-8e9a-beee5d141025 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:30.280045 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc50ed1-42bc-432d-8a45-8437d680acb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:30.280655 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc50ed1-42bc-432d-8a45-8437d680acb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:30.281042 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc50ed1-42bc-432d-8a45-8437d680acb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:30.281311 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc50ed1-42bc-432d-8a45-8437d680acb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:30.281949 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc50ed1-42bc-432d-8a45-8437d680acb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:18:30.282337 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc50ed1-42bc-432d-8a45-8437d680acb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:30.283919 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc50ed1-42bc-432d-8a45-8437d680acb3 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:30.290318 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ed17a-b381-4542-8e9a-beee5d141025 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:30.294814 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca146f-320a-4c8f-a2c0-d142c08c76eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:30.295402 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca146f-320a-4c8f-a2c0-d142c08c76eb admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:30.295676 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca146f-320a-4c8f-a2c0-d142c08c76eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:30.295943 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca146f-320a-4c8f-a2c0-d142c08c76eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:30.296368 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 793/1569] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:18:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:18:30.296579 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ca146f-320a-4c8f-a2c0-d142c08c76eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:30.297222 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc50ed1-42bc-432d-8a45-8437d680acb3 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:30.298242 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ca146f-320a-4c8f-a2c0-d142c08c76eb admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:30.309507 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 777/1570] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:18:30.309678 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca146f-320a-4c8f-a2c0-d142c08c76eb admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:30.313901 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc50ed1-42bc-432d-8a45-8437d680acb3 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:30.318577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0f8192-e4e4-4e80-9274-45064d7ead23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:30.319167 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca146f-320a-4c8f-a2c0-d142c08c76eb admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:30.319288 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f8192-e4e4-4e80-9274-45064d7ead23 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:30.319439 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f8192-e4e4-4e80-9274-45064d7ead23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:30.319731 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f8192-e4e4-4e80-9274-45064d7ead23 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:30.320389 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0f8192-e4e4-4e80-9274-45064d7ead23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:18:30.320688 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0f8192-e4e4-4e80-9274-45064d7ead23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:30.322317 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0f8192-e4e4-4e80-9274-45064d7ead23 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:30.323744 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b6a87f-b257-4e2b-8fa7-7b027ec33d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:30.324257 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b6a87f-b257-4e2b-8fa7-7b027ec33d2e admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:30.324509 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b6a87f-b257-4e2b-8fa7-7b027ec33d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:30.324803 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b6a87f-b257-4e2b-8fa7-7b027ec33d2e admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:30.325421 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b6a87f-b257-4e2b-8fa7-7b027ec33d2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:30.327327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b6a87f-b257-4e2b-8fa7-7b027ec33d2e admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:30.341337 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 794/1571] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:18:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:18:30.341337 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4bf9cc-f2cc-48ee-b85f-4f2f4feb5dfc None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:30.341337 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4bf9cc-f2cc-48ee-b85f-4f2f4feb5dfc None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:30.341337 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4bf9cc-f2cc-48ee-b85f-4f2f4feb5dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:30.359286 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 778/1572] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:18:30 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:18:30.359478 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-3f4bf9cc-f2cc-48ee-b85f-4f2f4feb5dfc None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:30.359597 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b6a87f-b257-4e2b-8fa7-7b027ec33d2e admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:30.359698 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3f4bf9cc-f2cc-48ee-b85f-4f2f4feb5dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:30.391059 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:30.417500 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 795/1573] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:30.419468 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b6a87f-b257-4e2b-8fa7-7b027ec33d2e admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:30.421606 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4bf9cc-f2cc-48ee-b85f-4f2f4feb5dfc None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:30.425150 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d285d-d01d-4afc-b8ce-b6be1b988ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:30.425863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d285d-d01d-4afc-b8ce-b6be1b988ce6 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:30.426224 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d285d-d01d-4afc-b8ce-b6be1b988ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:30.426439 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d285d-d01d-4afc-b8ce-b6be1b988ce6 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:30.427074 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d285d-d01d-4afc-b8ce-b6be1b988ce6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:30.429702 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d285d-d01d-4afc-b8ce-b6be1b988ce6 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:30.431324 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4bf9cc-f2cc-48ee-b85f-4f2f4feb5dfc None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:30.435919 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b35cb9-6dde-425f-8830-2e702f481334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:30.436570 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b35cb9-6dde-425f-8830-2e702f481334 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:30.436843 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b35cb9-6dde-425f-8830-2e702f481334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:30.437109 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b35cb9-6dde-425f-8830-2e702f481334 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:30.437906 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b35cb9-6dde-425f-8830-2e702f481334 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:18:30.438093 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b35cb9-6dde-425f-8830-2e702f481334 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:30.439705 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b35cb9-6dde-425f-8830-2e702f481334 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:30.459371 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 796/1574] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:18:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:18:30.459371 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b35cb9-6dde-425f-8830-2e702f481334 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:30.459723 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 779/1575] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:18:30 2022] POST /identity/v3/users => generated 302 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:18:30.459723 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d285d-d01d-4afc-b8ce-b6be1b988ce6 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:30.465655 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d285d-d01d-4afc-b8ce-b6be1b988ce6 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:30.469706 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b35cb9-6dde-425f-8830-2e702f481334 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:30.471791 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b913f87-7b39-4a28-9ea5-5f56d780cb53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:30.472387 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b913f87-7b39-4a28-9ea5-5f56d780cb53 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:30.472664 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b913f87-7b39-4a28-9ea5-5f56d780cb53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:30.472932 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b913f87-7b39-4a28-9ea5-5f56d780cb53 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:30.473560 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b913f87-7b39-4a28-9ea5-5f56d780cb53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:30.474392 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e73a3-34a6-4195-9968-d462fe664201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:30.474983 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e73a3-34a6-4195-9968-d462fe664201 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:30.475264 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e73a3-34a6-4195-9968-d462fe664201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:30.475524 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b913f87-7b39-4a28-9ea5-5f56d780cb53 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:30.475630 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e73a3-34a6-4195-9968-d462fe664201 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:30.476145 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590e73a3-34a6-4195-9968-d462fe664201 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:30.478205 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590e73a3-34a6-4195-9968-d462fe664201 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:30.489044 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:30.507066 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 780/1576] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:18:30 2022] POST /identity/v3/projects => generated 405 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:18:30.507828 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 797/1577] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:18:30 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:18:30.508255 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b913f87-7b39-4a28-9ea5-5f56d780cb53 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:30.511836 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e73a3-34a6-4195-9968-d462fe664201 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:30.577434 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e73a3-34a6-4195-9968-d462fe664201 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:30.579451 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b913f87-7b39-4a28-9ea5-5f56d780cb53 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:30.583732 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb26574-3ce5-4930-b5de-2c259f4782e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:30.584363 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb26574-3ce5-4930-b5de-2c259f4782e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:30.584585 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c40b6-bbb4-4290-99c4-7e7f6a0be833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:30.584710 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb26574-3ce5-4930-b5de-2c259f4782e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:30.584907 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb26574-3ce5-4930-b5de-2c259f4782e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:30.585328 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c40b6-bbb4-4290-99c4-7e7f6a0be833 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:30.585529 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb26574-3ce5-4930-b5de-2c259f4782e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:30.585815 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c40b6-bbb4-4290-99c4-7e7f6a0be833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:30.586340 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c40b6-bbb4-4290-99c4-7e7f6a0be833 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:30.587492 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb26574-3ce5-4930-b5de-2c259f4782e9 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:30.587615 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52c40b6-bbb4-4290-99c4-7e7f6a0be833 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:30.589618 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52c40b6-bbb4-4290-99c4-7e7f6a0be833 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:30.600678 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 781/1578] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:18:30.604418 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c40b6-bbb4-4290-99c4-7e7f6a0be833 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:30.615908 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c40b6-bbb4-4290-99c4-7e7f6a0be833 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:30.617174 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 798/1579] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:18:30 2022] POST /identity/v3/projects => generated 389 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:18:30.618049 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb26574-3ce5-4930-b5de-2c259f4782e9 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:30.676117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80363121-0c91-44f6-b557-63dfde185edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:30.676728 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80363121-0c91-44f6-b557-63dfde185edf admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:30.676935 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb26574-3ce5-4930-b5de-2c259f4782e9 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:30.677077 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80363121-0c91-44f6-b557-63dfde185edf admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:30.677269 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80363121-0c91-44f6-b557-63dfde185edf admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:30.677911 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80363121-0c91-44f6-b557-63dfde185edf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:30.679881 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80363121-0c91-44f6-b557-63dfde185edf admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:30.680536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4f1a3-c1ab-47bd-bb9f-152a27a3289b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:30.681138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4f1a3-c1ab-47bd-bb9f-152a27a3289b service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:30.681447 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4f1a3-c1ab-47bd-bb9f-152a27a3289b service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:30.681710 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4f1a3-c1ab-47bd-bb9f-152a27a3289b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:30.690687 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc4f1a3-c1ab-47bd-bb9f-152a27a3289b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:30.692359 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc4f1a3-c1ab-47bd-bb9f-152a27a3289b service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:30.702106 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 782/1580] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:18:30 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:18:30.702751 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 799/1581] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:18:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:18:30.702934 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80363121-0c91-44f6-b557-63dfde185edf admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:30.703495 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4f1a3-c1ab-47bd-bb9f-152a27a3289b service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:30.713528 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4f1a3-c1ab-47bd-bb9f-152a27a3289b service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:30.713779 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80363121-0c91-44f6-b557-63dfde185edf admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:30.719190 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22195e6-a873-42c6-a817-bc9cfd9fa195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:30.719812 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22195e6-a873-42c6-a817-bc9cfd9fa195 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:30.720086 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22195e6-a873-42c6-a817-bc9cfd9fa195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:30.720197 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a3f7e-dda5-4934-907e-804dd23df921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:30.720379 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22195e6-a873-42c6-a817-bc9cfd9fa195 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:30.720750 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a3f7e-dda5-4934-907e-804dd23df921 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:30.721010 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22195e6-a873-42c6-a817-bc9cfd9fa195 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:30.721131 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a3f7e-dda5-4934-907e-804dd23df921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:30.721330 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a3f7e-dda5-4934-907e-804dd23df921 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:30.722116 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a3f7e-dda5-4934-907e-804dd23df921 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:30.723075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22195e6-a873-42c6-a817-bc9cfd9fa195 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:30.724243 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a3f7e-dda5-4934-907e-804dd23df921 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:30.738826 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 783/1582] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:18:30.739659 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a3f7e-dda5-4934-907e-804dd23df921 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:30.750090 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a3f7e-dda5-4934-907e-804dd23df921 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:30.755708 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c2c22-c4da-4607-a287-0e2932c9b94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:30.756007 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c2c22-c4da-4607-a287-0e2932c9b94c admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:30.756294 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c2c22-c4da-4607-a287-0e2932c9b94c admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:30.756561 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c2c22-c4da-4607-a287-0e2932c9b94c admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:30.757294 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020c2c22-c4da-4607-a287-0e2932c9b94c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:30.761379 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 800/1583] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:18:30 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:18:30.761379 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22195e6-a873-42c6-a817-bc9cfd9fa195 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:30.761579 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020c2c22-c4da-4607-a287-0e2932c9b94c admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:30.776605 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:30.787286 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 784/1584] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:18:30 2022] POST /identity/v3/users => generated 303 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:18:30.788741 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c2c22-c4da-4607-a287-0e2932c9b94c admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:30.803398 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c2c22-c4da-4607-a287-0e2932c9b94c admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:30.830225 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22195e6-a873-42c6-a817-bc9cfd9fa195 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:30.835140 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd77cb-458f-4058-a7b1-28ac7006e7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:30.835700 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd77cb-458f-4058-a7b1-28ac7006e7df admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:30.836025 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd77cb-458f-4058-a7b1-28ac7006e7df admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:30.836289 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd77cb-458f-4058-a7b1-28ac7006e7df admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:30.836975 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedd77cb-458f-4058-a7b1-28ac7006e7df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:30.839237 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedd77cb-458f-4058-a7b1-28ac7006e7df admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:30.858684 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:30.865607 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 801/1585] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:18:30 2022] POST /identity/v3/users => generated 287 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:18:30.869364 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd77cb-458f-4058-a7b1-28ac7006e7df admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:30.873592 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c85f9e-8a7e-4062-b13b-3f18e8d33b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:30.874254 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c85f9e-8a7e-4062-b13b-3f18e8d33b79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:30.874871 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c85f9e-8a7e-4062-b13b-3f18e8d33b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:30.875154 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c85f9e-8a7e-4062-b13b-3f18e8d33b79 admin admin] PATH_INFO: `/v3/projects/8ae84d5c59f0431887b06223cb120691/users/9a56feccbe844aff84e2b35a179a9e16/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:30.880991 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd77cb-458f-4058-a7b1-28ac7006e7df admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:30.887643 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb59a61-806d-4bbe-8c08-031a7924cc06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:30.888850 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb59a61-806d-4bbe-8c08-031a7924cc06 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:30.889410 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb59a61-806d-4bbe-8c08-031a7924cc06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:30.893186 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb59a61-806d-4bbe-8c08-031a7924cc06 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:30.893665 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c85f9e-8a7e-4062-b13b-3f18e8d33b79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ae84d5c59f0431887b06223cb120691, user_id=9a56feccbe844aff84e2b35a179a9e16, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:30.894750 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb59a61-806d-4bbe-8c08-031a7924cc06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:30.894989 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: WARNING py.warnings [None req-43c85f9e-8a7e-4062-b13b-3f18e8d33b79 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 05:18:30.894989 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: warnings.warn(msg) Aug 31 05:18:30.894989 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: Aug 31 05:18:30.897819 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c85f9e-8a7e-4062-b13b-3f18e8d33b79 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:30.898653 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb59a61-806d-4bbe-8c08-031a7924cc06 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:30.920692 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 785/1586] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:18:30 2022] PUT /identity/v3/projects/8ae84d5c59f0431887b06223cb120691/users/9a56feccbe844aff84e2b35a179a9e16/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:30.921301 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c85f9e-8a7e-4062-b13b-3f18e8d33b79 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:30.929549 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 802/1587] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:18:30 2022] POST /identity/v3/users => generated 295 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:18:30.931763 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb59a61-806d-4bbe-8c08-031a7924cc06 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:30.943756 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb59a61-806d-4bbe-8c08-031a7924cc06 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:30.996097 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c85f9e-8a7e-4062-b13b-3f18e8d33b79 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:31.002184 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589255a2-8192-4d97-b172-14d78c81d93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:31.002805 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589255a2-8192-4d97-b172-14d78c81d93d admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:31.003319 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589255a2-8192-4d97-b172-14d78c81d93d admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:31.003319 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589255a2-8192-4d97-b172-14d78c81d93d admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:31.004043 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589255a2-8192-4d97-b172-14d78c81d93d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:31.005659 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589255a2-8192-4d97-b172-14d78c81d93d admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:31.009627 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff34f740-b93d-47cf-90da-c49daafcacdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:31.010267 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34f740-b93d-47cf-90da-c49daafcacdf admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:31.010661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34f740-b93d-47cf-90da-c49daafcacdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:31.010937 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34f740-b93d-47cf-90da-c49daafcacdf admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:31.012020 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff34f740-b93d-47cf-90da-c49daafcacdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:31.013689 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff34f740-b93d-47cf-90da-c49daafcacdf admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:31.016452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 786/1588] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:18:31.017391 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589255a2-8192-4d97-b172-14d78c81d93d admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:31.028017 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 803/1589] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:18:31.028191 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589255a2-8192-4d97-b172-14d78c81d93d admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:31.031728 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff34f740-b93d-47cf-90da-c49daafcacdf admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:31.037656 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343af946-d549-4e14-9bc6-c8ce472c8442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:31.038890 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343af946-d549-4e14-9bc6-c8ce472c8442 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:31.039176 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343af946-d549-4e14-9bc6-c8ce472c8442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:31.039455 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343af946-d549-4e14-9bc6-c8ce472c8442 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:31.040114 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343af946-d549-4e14-9bc6-c8ce472c8442 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:31.042267 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343af946-d549-4e14-9bc6-c8ce472c8442 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:31.044368 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff34f740-b93d-47cf-90da-c49daafcacdf admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:31.052492 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed90a286-4e21-4ead-a389-a3a4d7955060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:31.053648 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed90a286-4e21-4ead-a389-a3a4d7955060 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:31.054739 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed90a286-4e21-4ead-a389-a3a4d7955060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:31.055026 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed90a286-4e21-4ead-a389-a3a4d7955060 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:31.055765 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed90a286-4e21-4ead-a389-a3a4d7955060 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:31.057364 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed90a286-4e21-4ead-a389-a3a4d7955060 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:31.073403 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 804/1590] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:18:31.077355 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 787/1591] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:18:31 2022] POST /identity/v3/users => generated 287 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:18:31.078359 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed90a286-4e21-4ead-a389-a3a4d7955060 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:31.081788 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e631b43-332e-4854-8c8d-cacb7f73cccc None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:31.082211 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e631b43-332e-4854-8c8d-cacb7f73cccc None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:31.082488 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e631b43-332e-4854-8c8d-cacb7f73cccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:31.099551 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed90a286-4e21-4ead-a389-a3a4d7955060 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:31.104331 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f1c7de-85be-403a-af5d-fce421852461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:31.104953 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1c7de-85be-403a-af5d-fce421852461 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:31.105231 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1c7de-85be-403a-af5d-fce421852461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:31.105502 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1c7de-85be-403a-af5d-fce421852461 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:31.106183 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f1c7de-85be-403a-af5d-fce421852461 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:31.107955 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f1c7de-85be-403a-af5d-fce421852461 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:31.122990 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 805/1592] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:18:31.122990 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f1c7de-85be-403a-af5d-fce421852461 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:31.125350 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-4e631b43-332e-4854-8c8d-cacb7f73cccc None None] MFA Rules not processed for user `9a56feccbe844aff84e2b35a179a9e16`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:31.127511 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-4e631b43-332e-4854-8c8d-cacb7f73cccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:31.132055 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f1c7de-85be-403a-af5d-fce421852461 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:31.137278 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8289adc-5189-4e91-82ba-5bd982ce81a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:31.138078 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8289adc-5189-4e91-82ba-5bd982ce81a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:31.138362 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8289adc-5189-4e91-82ba-5bd982ce81a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:31.138634 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8289adc-5189-4e91-82ba-5bd982ce81a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:31.139281 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8289adc-5189-4e91-82ba-5bd982ce81a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:31.141079 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8289adc-5189-4e91-82ba-5bd982ce81a0 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:31.155771 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 806/1593] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:18:31.156665 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8289adc-5189-4e91-82ba-5bd982ce81a0 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:31.169456 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8289adc-5189-4e91-82ba-5bd982ce81a0 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:31.175315 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f64143-aa07-41b0-8065-b512a690f33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:31.176023 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f64143-aa07-41b0-8065-b512a690f33e admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:31.176490 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f64143-aa07-41b0-8065-b512a690f33e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:31.176765 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f64143-aa07-41b0-8065-b512a690f33e admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:31.177461 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f64143-aa07-41b0-8065-b512a690f33e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:31.179548 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f64143-aa07-41b0-8065-b512a690f33e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:31.191863 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 807/1594] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:18:31.191863 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f64143-aa07-41b0-8065-b512a690f33e admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:31.203501 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f64143-aa07-41b0-8065-b512a690f33e admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:31.209106 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02793888-1726-46d2-8927-0d1ad07a246b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:31.209783 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02793888-1726-46d2-8927-0d1ad07a246b admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:31.210085 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02793888-1726-46d2-8927-0d1ad07a246b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:31.210577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02793888-1726-46d2-8927-0d1ad07a246b admin admin] PATH_INFO: `/v3/projects/e5368d3255474cd08e31d3c2bd952cc1/users/5d566ef3f774464189527523acbe908c/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:31.222142 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 788/1595] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:31.222142 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e631b43-332e-4854-8c8d-cacb7f73cccc None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:31.227320 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02793888-1726-46d2-8927-0d1ad07a246b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5368d3255474cd08e31d3c2bd952cc1, user_id=5d566ef3f774464189527523acbe908c, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:31.229993 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02793888-1726-46d2-8927-0d1ad07a246b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:31.243681 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e631b43-332e-4854-8c8d-cacb7f73cccc None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:31.248718 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 808/1596] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:18:31 2022] PUT /identity/v3/projects/e5368d3255474cd08e31d3c2bd952cc1/users/5d566ef3f774464189527523acbe908c/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:31.250159 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02793888-1726-46d2-8927-0d1ad07a246b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:31.266934 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:31.296605 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:31.317606 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1a4c3-4c5e-4d96-ab8b-65df4fe8624c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:31.318258 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1a4c3-4c5e-4d96-ab8b-65df4fe8624c admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:31.318560 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1a4c3-4c5e-4d96-ab8b-65df4fe8624c admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:31.318823 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1a4c3-4c5e-4d96-ab8b-65df4fe8624c admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:31.319472 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd1a4c3-4c5e-4d96-ab8b-65df4fe8624c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:31.321073 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd1a4c3-4c5e-4d96-ab8b-65df4fe8624c admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:31.325136 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02793888-1726-46d2-8927-0d1ad07a246b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:31.330414 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b4134d-ada3-4266-ae4f-55a1d2fc7dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:31.331225 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b4134d-ada3-4266-ae4f-55a1d2fc7dc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:31.331705 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b4134d-ada3-4266-ae4f-55a1d2fc7dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:31.331879 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 789/1597] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:18:31.332355 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b4134d-ada3-4266-ae4f-55a1d2fc7dc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:31.333261 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b4134d-ada3-4266-ae4f-55a1d2fc7dc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:31.335624 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b4134d-ada3-4266-ae4f-55a1d2fc7dc6 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:31.335759 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1a4c3-4c5e-4d96-ab8b-65df4fe8624c admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:31.345782 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1a4c3-4c5e-4d96-ab8b-65df4fe8624c admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:31.350963 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 809/1598] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:18:31.350963 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b4134d-ada3-4266-ae4f-55a1d2fc7dc6 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:31.353354 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c7ce9-c247-4d8b-af2f-28cd1c6ddf2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:31.354121 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c7ce9-c247-4d8b-af2f-28cd1c6ddf2c admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:31.354581 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c7ce9-c247-4d8b-af2f-28cd1c6ddf2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:31.354847 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c7ce9-c247-4d8b-af2f-28cd1c6ddf2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:31.355589 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c7ce9-c247-4d8b-af2f-28cd1c6ddf2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:31.357320 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c7ce9-c247-4d8b-af2f-28cd1c6ddf2c admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:31.359898 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b4134d-ada3-4266-ae4f-55a1d2fc7dc6 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:31.368820 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d040a4-5322-42ac-8ed5-86827a4fe533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:31.369522 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d040a4-5322-42ac-8ed5-86827a4fe533 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:31.369972 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d040a4-5322-42ac-8ed5-86827a4fe533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:31.370441 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d040a4-5322-42ac-8ed5-86827a4fe533 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:31.371848 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d040a4-5322-42ac-8ed5-86827a4fe533 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:31.373123 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d040a4-5322-42ac-8ed5-86827a4fe533 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:31.375906 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 790/1599] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:18:31.376740 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c7ce9-c247-4d8b-af2f-28cd1c6ddf2c admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:31.390113 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 810/1600] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:18:31.390346 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c7ce9-c247-4d8b-af2f-28cd1c6ddf2c admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:31.390489 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ca233-50d0-4b29-98ba-f3cf4ca02309 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:31.390489 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ca233-50d0-4b29-98ba-f3cf4ca02309 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:31.390489 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ca233-50d0-4b29-98ba-f3cf4ca02309 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:31.391881 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 811/1601] 10.222.0.36 () {54 vars in 962 bytes} [Wed Aug 31 05:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:18:31.393096 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195cdb4-4750-4714-9968-60f99c12d086 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:31.393472 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195cdb4-4750-4714-9968-60f99c12d086 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:31.393830 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195cdb4-4750-4714-9968-60f99c12d086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:31.394846 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e71d463-4a9d-4ee8-8d1f-fa4a050c2261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:31.395677 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e71d463-4a9d-4ee8-8d1f-fa4a050c2261 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:31.396119 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e71d463-4a9d-4ee8-8d1f-fa4a050c2261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:31.396530 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e71d463-4a9d-4ee8-8d1f-fa4a050c2261 admin admin] PATH_INFO: `/v3/projects/14058365e648441d96bb725c33f80923/users/4b07cff5bc3a4e68b62c4c5d3e798732/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:31.416042 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-1195cdb4-4750-4714-9968-60f99c12d086 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:31.417476 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1195cdb4-4750-4714-9968-60f99c12d086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:31.418539 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e71d463-4a9d-4ee8-8d1f-fa4a050c2261 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14058365e648441d96bb725c33f80923, user_id=4b07cff5bc3a4e68b62c4c5d3e798732, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:31.421440 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e71d463-4a9d-4ee8-8d1f-fa4a050c2261 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:31.427708 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 812/1602] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:31.428543 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1195cdb4-4750-4714-9968-60f99c12d086 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:31.439869 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1195cdb4-4750-4714-9968-60f99c12d086 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:31.444913 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 791/1603] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:18:31 2022] PUT /identity/v3/projects/14058365e648441d96bb725c33f80923/users/4b07cff5bc3a4e68b62c4c5d3e798732/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:31.445909 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e71d463-4a9d-4ee8-8d1f-fa4a050c2261 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:31.525735 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e71d463-4a9d-4ee8-8d1f-fa4a050c2261 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:31.528800 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2349c192-70be-43d0-aa7e-ed89847bef60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:31.529935 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349c192-70be-43d0-aa7e-ed89847bef60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:31.530556 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349c192-70be-43d0-aa7e-ed89847bef60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:31.531033 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2062127e-a3fa-402a-9034-dcdab1293e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:31.531391 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349c192-70be-43d0-aa7e-ed89847bef60 admin admin] PATH_INFO: `/v3/projects/ac903c54f50c490cbf4aacf457bed01b/users/1f6e9fa5046c4b179b49d3efbe3845ed/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:31.531770 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062127e-a3fa-402a-9034-dcdab1293e72 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:31.532385 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062127e-a3fa-402a-9034-dcdab1293e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:31.532671 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062127e-a3fa-402a-9034-dcdab1293e72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:31.535617 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2062127e-a3fa-402a-9034-dcdab1293e72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:31.537255 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2062127e-a3fa-402a-9034-dcdab1293e72 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:31.549493 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 792/1604] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:18:31.550643 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2062127e-a3fa-402a-9034-dcdab1293e72 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:31.557456 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2349c192-70be-43d0-aa7e-ed89847bef60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac903c54f50c490cbf4aacf457bed01b, user_id=1f6e9fa5046c4b179b49d3efbe3845ed, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:31.561960 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2349c192-70be-43d0-aa7e-ed89847bef60 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:31.567912 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2062127e-a3fa-402a-9034-dcdab1293e72 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:31.573952 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853bb7f9-9efe-43bc-943a-5ad43511deef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:31.574663 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bb7f9-9efe-43bc-943a-5ad43511deef admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:31.575109 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bb7f9-9efe-43bc-943a-5ad43511deef admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:31.575440 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bb7f9-9efe-43bc-943a-5ad43511deef admin admin] PATH_INFO: `/v3/projects/99ca2cb2b3c44c2a8bfec89fe42b8387/users/5ed9b41695f746868768127dad722f7e/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:31.584327 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 813/1605] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:18:31 2022] PUT /identity/v3/projects/ac903c54f50c490cbf4aacf457bed01b/users/1f6e9fa5046c4b179b49d3efbe3845ed/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:31.591548 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2349c192-70be-43d0-aa7e-ed89847bef60 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:31.593913 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853bb7f9-9efe-43bc-943a-5ad43511deef admin admin] RBAC: Authorizing `identity:create_grant(project_id=99ca2cb2b3c44c2a8bfec89fe42b8387, user_id=5ed9b41695f746868768127dad722f7e, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:31.597461 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853bb7f9-9efe-43bc-943a-5ad43511deef admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:31.624295 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 793/1606] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:18:31 2022] PUT /identity/v3/projects/99ca2cb2b3c44c2a8bfec89fe42b8387/users/5ed9b41695f746868768127dad722f7e/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:31.624295 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0d19d-d248-408d-b36a-b1353e971715 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:31.624295 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0d19d-d248-408d-b36a-b1353e971715 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:31.624295 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0d19d-d248-408d-b36a-b1353e971715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:31.661791 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-65b0d19d-d248-408d-b36a-b1353e971715 None None] MFA Rules not processed for user `f27665b1574a4db7b7a9771f7d775826`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:31.663406 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-65b0d19d-d248-408d-b36a-b1353e971715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:31.700404 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2349c192-70be-43d0-aa7e-ed89847bef60 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:31.728443 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:31.731796 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:31.755330 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 794/1607] 10.222.0.36 () {58 vars in 1059 bytes} [Wed Aug 31 05:18:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:31.756554 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b0d19d-d248-408d-b36a-b1353e971715 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:31.784914 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8076c43-cb27-44f4-80ca-3fcfed47f1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:31.785969 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8076c43-cb27-44f4-80ca-3fcfed47f1b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:31.786412 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8076c43-cb27-44f4-80ca-3fcfed47f1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:31.786842 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8076c43-cb27-44f4-80ca-3fcfed47f1b7 admin admin] PATH_INFO: `/v3/projects/164c57b95c274f89bd391cf111efa374/users/af50ca6347404cdf8eb213d27053af16/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:31.804390 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8076c43-cb27-44f4-80ca-3fcfed47f1b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=164c57b95c274f89bd391cf111efa374, user_id=af50ca6347404cdf8eb213d27053af16, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:31.807179 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8076c43-cb27-44f4-80ca-3fcfed47f1b7 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:31.827820 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 814/1608] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:18:31 2022] PUT /identity/v3/projects/164c57b95c274f89bd391cf111efa374/users/af50ca6347404cdf8eb213d27053af16/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:31.829384 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d2953-29ad-483f-858d-ed9a144beed6 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:31.829784 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d2953-29ad-483f-858d-ed9a144beed6 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:31.830205 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d2953-29ad-483f-858d-ed9a144beed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:31.850107 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b0d19d-d248-408d-b36a-b1353e971715 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:31.855504 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f42d2953-29ad-483f-858d-ed9a144beed6 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:31.857459 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f42d2953-29ad-483f-858d-ed9a144beed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:31.918668 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc7459d-e29d-49c4-9f3e-8f3fc4966d3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:31.919390 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7459d-e29d-49c4-9f3e-8f3fc4966d3f service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:31.919707 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7459d-e29d-49c4-9f3e-8f3fc4966d3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:31.919992 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7459d-e29d-49c4-9f3e-8f3fc4966d3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:31.933315 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 815/1609] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:31.937391 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42d2953-29ad-483f-858d-ed9a144beed6 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:31.945740 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42d2953-29ad-483f-858d-ed9a144beed6 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:31.957989 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d20643-e1dc-4d14-b597-7aab16f6e247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:31.958785 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d20643-e1dc-4d14-b597-7aab16f6e247 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:31.959066 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d20643-e1dc-4d14-b597-7aab16f6e247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:31.959382 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d20643-e1dc-4d14-b597-7aab16f6e247 admin admin] PATH_INFO: `/v3/projects/7c7f893cbdb04887bfc04fcc56da939e/users/fd7ec99ba8e0464d98840d5f4a38417e/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:31.985047 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d20643-e1dc-4d14-b597-7aab16f6e247 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c7f893cbdb04887bfc04fcc56da939e, user_id=fd7ec99ba8e0464d98840d5f4a38417e, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:31.988554 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d20643-e1dc-4d14-b597-7aab16f6e247 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:32.011178 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc7459d-e29d-49c4-9f3e-8f3fc4966d3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:32.013637 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc7459d-e29d-49c4-9f3e-8f3fc4966d3f service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:32.018771 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 816/1610] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:18:31 2022] PUT /identity/v3/projects/7c7f893cbdb04887bfc04fcc56da939e/users/fd7ec99ba8e0464d98840d5f4a38417e/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:32.020174 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319c16a-0ff4-4e8d-8a40-9fa1a66b6e3b None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:32.020522 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319c16a-0ff4-4e8d-8a40-9fa1a66b6e3b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:32.020716 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319c16a-0ff4-4e8d-8a40-9fa1a66b6e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:32.039615 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 795/1611] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:18:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:18:32.041450 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed9024-10d1-4789-b14a-84d1655428b9 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:32.042772 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed9024-10d1-4789-b14a-84d1655428b9 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:32.045647 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed9024-10d1-4789-b14a-84d1655428b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:32.060772 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-3319c16a-0ff4-4e8d-8a40-9fa1a66b6e3b None None] MFA Rules not processed for user `1f6e9fa5046c4b179b49d3efbe3845ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:32.062671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3319c16a-0ff4-4e8d-8a40-9fa1a66b6e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:32.121892 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-c4ed9024-10d1-4789-b14a-84d1655428b9 None None] MFA Rules not processed for user `5ed9b41695f746868768127dad722f7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:32.123523 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c4ed9024-10d1-4789-b14a-84d1655428b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:32.170936 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 817/1612] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:32 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:32.177323 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319c16a-0ff4-4e8d-8a40-9fa1a66b6e3b None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:32.195710 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:32.227685 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:32.266226 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 796/1613] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:32.268022 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d93d5c-0c8b-4a49-8698-a28ab9394adb None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:32.268408 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d93d5c-0c8b-4a49-8698-a28ab9394adb None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:32.268878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d93d5c-0c8b-4a49-8698-a28ab9394adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:32.307460 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319c16a-0ff4-4e8d-8a40-9fa1a66b6e3b None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:32.308174 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-d3d93d5c-0c8b-4a49-8698-a28ab9394adb None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:32.309389 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-d3d93d5c-0c8b-4a49-8698-a28ab9394adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:32.320352 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1342af6a-4dfd-4315-aeff-e4edf8018e70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:32.320991 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1342af6a-4dfd-4315-aeff-e4edf8018e70 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:32.321270 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1342af6a-4dfd-4315-aeff-e4edf8018e70 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:32.321543 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1342af6a-4dfd-4315-aeff-e4edf8018e70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:32.437457 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 797/1614] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:32.438185 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d93d5c-0c8b-4a49-8698-a28ab9394adb None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:32.438609 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1342af6a-4dfd-4315-aeff-e4edf8018e70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:32.442261 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1342af6a-4dfd-4315-aeff-e4edf8018e70 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:32.456391 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 818/1615] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:18:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:18:32.457887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e69a7a-ee6c-4c80-b45b-2357dbbf006a None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:32.458289 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e69a7a-ee6c-4c80-b45b-2357dbbf006a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:32.458654 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e69a7a-ee6c-4c80-b45b-2357dbbf006a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:32.492879 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-89e69a7a-ee6c-4c80-b45b-2357dbbf006a None None] MFA Rules not processed for user `fd7ec99ba8e0464d98840d5f4a38417e`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:32.494501 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-89e69a7a-ee6c-4c80-b45b-2357dbbf006a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:32.563828 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d93d5c-0c8b-4a49-8698-a28ab9394adb None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:32.577326 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a1258-dd97-417b-b232-bb2def363230 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:32.578538 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a1258-dd97-417b-b232-bb2def363230 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:32.579060 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a1258-dd97-417b-b232-bb2def363230 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:32.579477 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a1258-dd97-417b-b232-bb2def363230 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:32.692620 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5a1258-dd97-417b-b232-bb2def363230 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:32.692620 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5a1258-dd97-417b-b232-bb2def363230 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:32.692620 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 798/1616] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:18:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:18:32.692620 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a1258-dd97-417b-b232-bb2def363230 service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:32.692620 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a1258-dd97-417b-b232-bb2def363230 service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:32.693478 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 819/1617] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:32.693478 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e69a7a-ee6c-4c80-b45b-2357dbbf006a None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:32.693797 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871ecabd-586f-4c49-890f-50db2ea4189b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:32.693797 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ecabd-586f-4c49-890f-50db2ea4189b admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:32.693797 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ecabd-586f-4c49-890f-50db2ea4189b admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:32.693797 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ecabd-586f-4c49-890f-50db2ea4189b admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:32.694244 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871ecabd-586f-4c49-890f-50db2ea4189b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:32.694244 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871ecabd-586f-4c49-890f-50db2ea4189b admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:32.734631 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e69a7a-ee6c-4c80-b45b-2357dbbf006a None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:32.758262 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44182d17-d303-4883-a960-2ef520e13315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:32.759482 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44182d17-d303-4883-a960-2ef520e13315 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:32.760984 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44182d17-d303-4883-a960-2ef520e13315 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:32.761489 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44182d17-d303-4883-a960-2ef520e13315 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:32.763742 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44182d17-d303-4883-a960-2ef520e13315 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:32.766689 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 799/1618] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:18:32 2022] POST /identity/v3/projects => generated 405 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:18:32.767749 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871ecabd-586f-4c49-890f-50db2ea4189b admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:32.769627 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44182d17-d303-4883-a960-2ef520e13315 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:32.833910 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 820/1619] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:18:32 2022] POST /identity/v3/projects => generated 373 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:18:32.833910 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e6c29-a3b7-48d9-9281-e123ca2b519a None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:32.833910 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e6c29-a3b7-48d9-9281-e123ca2b519a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:32.833910 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e6c29-a3b7-48d9-9281-e123ca2b519a None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:32.838201 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 821/1620] 10.222.0.36 () {54 vars in 962 bytes} [Wed Aug 31 05:18:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:18:32.838979 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e6c29-a3b7-48d9-9281-e123ca2b519a None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:32.869273 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:32.873444 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871ecabd-586f-4c49-890f-50db2ea4189b admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:32.908061 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:32.941058 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e6c29-a3b7-48d9-9281-e123ca2b519a None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:32.948000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fe2893-296b-40b9-9cb9-8a483ded4de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:32.949174 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe2893-296b-40b9-9cb9-8a483ded4de2 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:32.949720 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe2893-296b-40b9-9cb9-8a483ded4de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:32.949999 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe2893-296b-40b9-9cb9-8a483ded4de2 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:32.951049 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fe2893-296b-40b9-9cb9-8a483ded4de2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:32.953424 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fe2893-296b-40b9-9cb9-8a483ded4de2 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:32.955081 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb98800d-727a-4f9f-a6cd-87242553b8cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:32.956707 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb98800d-727a-4f9f-a6cd-87242553b8cb service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:32.957532 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb98800d-727a-4f9f-a6cd-87242553b8cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:32.958464 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb98800d-727a-4f9f-a6cd-87242553b8cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:32.971844 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb98800d-727a-4f9f-a6cd-87242553b8cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:32.976112 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb98800d-727a-4f9f-a6cd-87242553b8cb service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:32.992428 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 822/1621] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:18:32 2022] POST /identity/v3/users => generated 309 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:18:32.994399 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fe2893-296b-40b9-9cb9-8a483ded4de2 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:32.997851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 800/1622] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:18:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:18:33.005184 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa618db-1dd9-44a5-bb43-ae27288a27d1 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:33.006328 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa618db-1dd9-44a5-bb43-ae27288a27d1 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:33.006790 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa618db-1dd9-44a5-bb43-ae27288a27d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:33.017733 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fe2893-296b-40b9-9cb9-8a483ded4de2 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:33.023575 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8be4c-9ba9-4347-b36d-386e718928e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:33.024529 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8be4c-9ba9-4347-b36d-386e718928e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:33.024962 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8be4c-9ba9-4347-b36d-386e718928e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:33.025312 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8be4c-9ba9-4347-b36d-386e718928e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:33.026291 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8be4c-9ba9-4347-b36d-386e718928e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:33.028625 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8be4c-9ba9-4347-b36d-386e718928e4 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:33.032896 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-1fa618db-1dd9-44a5-bb43-ae27288a27d1 None None] MFA Rules not processed for user `f27665b1574a4db7b7a9771f7d775826`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:33.034604 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-1fa618db-1dd9-44a5-bb43-ae27288a27d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:33.062379 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 823/1623] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:18:32 2022] POST /identity/v3/users => generated 293 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:18:33.063455 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8be4c-9ba9-4347-b36d-386e718928e4 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:33.077775 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8be4c-9ba9-4347-b36d-386e718928e4 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:33.084008 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62d455-9d42-48d4-8f0b-a546cea145ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:33.096028 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62d455-9d42-48d4-8f0b-a546cea145ef admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:33.097515 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62d455-9d42-48d4-8f0b-a546cea145ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:33.103775 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62d455-9d42-48d4-8f0b-a546cea145ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:33.104730 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b62d455-9d42-48d4-8f0b-a546cea145ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:33.106470 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b62d455-9d42-48d4-8f0b-a546cea145ef admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:33.122775 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 824/1624] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:18:33.124508 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf203c0-e6ec-45ce-960e-91bca257968d None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:33.124798 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf203c0-e6ec-45ce-960e-91bca257968d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:33.125069 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf203c0-e6ec-45ce-960e-91bca257968d None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:33.128334 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 825/1625] 10.222.0.36 () {54 vars in 912 bytes} [Wed Aug 31 05:18:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:18:33.129158 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf203c0-e6ec-45ce-960e-91bca257968d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:33.143378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf203c0-e6ec-45ce-960e-91bca257968d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:33.151003 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b0daed-360f-414f-a082-cfd17f61a9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:33.151662 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0daed-360f-414f-a082-cfd17f61a9b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:33.152163 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0daed-360f-414f-a082-cfd17f61a9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:33.152386 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 801/1626] 10.222.0.36 () {58 vars in 1059 bytes} [Wed Aug 31 05:18:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:33.152527 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0daed-360f-414f-a082-cfd17f61a9b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:33.153179 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b0daed-360f-414f-a082-cfd17f61a9b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:33.153601 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa618db-1dd9-44a5-bb43-ae27288a27d1 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:33.155029 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b0daed-360f-414f-a082-cfd17f61a9b8 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:33.168860 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa618db-1dd9-44a5-bb43-ae27288a27d1 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:33.172974 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 826/1627] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:18:33.174455 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaef2bc-f5d1-4f6b-8d1b-f367bacc73ff None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:33.174749 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaef2bc-f5d1-4f6b-8d1b-f367bacc73ff None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:33.175041 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaef2bc-f5d1-4f6b-8d1b-f367bacc73ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:33.179651 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffef4d33-c6c3-476e-bd1f-e70ebd394d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:33.180724 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffef4d33-c6c3-476e-bd1f-e70ebd394d2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:33.181214 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffef4d33-c6c3-476e-bd1f-e70ebd394d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:33.181602 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffef4d33-c6c3-476e-bd1f-e70ebd394d2f admin admin] PATH_INFO: `/v3/projects/4a3839bf23f645c19a567b642c7e3238/users/1fd72285ddf645049ae486a03a986567/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:33.206274 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffef4d33-c6c3-476e-bd1f-e70ebd394d2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a3839bf23f645c19a567b642c7e3238, user_id=1fd72285ddf645049ae486a03a986567, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:33.207966 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-eeaef2bc-f5d1-4f6b-8d1b-f367bacc73ff None None] MFA Rules not processed for user `f27665b1574a4db7b7a9771f7d775826`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:33.209177 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffef4d33-c6c3-476e-bd1f-e70ebd394d2f admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:33.209808 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-eeaef2bc-f5d1-4f6b-8d1b-f367bacc73ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:33.225812 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 827/1628] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Aug 31 05:18:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:33.226859 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaef2bc-f5d1-4f6b-8d1b-f367bacc73ff None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:33.245160 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaef2bc-f5d1-4f6b-8d1b-f367bacc73ff None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:33.250898 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 802/1629] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:18:33 2022] PUT /identity/v3/projects/4a3839bf23f645c19a567b642c7e3238/users/1fd72285ddf645049ae486a03a986567/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:33.252025 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffef4d33-c6c3-476e-bd1f-e70ebd394d2f admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:33.365505 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cebf97a-2f73-4101-a6fa-25ef307904e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:33.366276 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cebf97a-2f73-4101-a6fa-25ef307904e4 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:33.366628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cebf97a-2f73-4101-a6fa-25ef307904e4 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:33.366960 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cebf97a-2f73-4101-a6fa-25ef307904e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:33.413335 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffef4d33-c6c3-476e-bd1f-e70ebd394d2f admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:33.428225 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c6015-dd11-4003-9812-10232a61c539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:33.430996 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c6015-dd11-4003-9812-10232a61c539 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:33.430996 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c6015-dd11-4003-9812-10232a61c539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:33.432135 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c6015-dd11-4003-9812-10232a61c539 admin admin] PATH_INFO: `/v3/projects/382300da8cf0482a96ab096871b75675/users/e7b35249c1544746bd8d4340821cf30b/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:33.454886 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:33.459961 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cebf97a-2f73-4101-a6fa-25ef307904e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:33.461958 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cebf97a-2f73-4101-a6fa-25ef307904e4 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:33.462550 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7c6015-dd11-4003-9812-10232a61c539 admin admin] RBAC: Authorizing `identity:create_grant(project_id=382300da8cf0482a96ab096871b75675, user_id=e7b35249c1544746bd8d4340821cf30b, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:33.465293 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7c6015-dd11-4003-9812-10232a61c539 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:33.490597 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 803/1630] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:18:33 2022] PUT /identity/v3/projects/382300da8cf0482a96ab096871b75675/users/e7b35249c1544746bd8d4340821cf30b/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:33.490597 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c6015-dd11-4003-9812-10232a61c539 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:33.490942 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 828/1631] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:18:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:18:33.493139 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cebf97a-2f73-4101-a6fa-25ef307904e4 service nova] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:33.517203 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:33.535307 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cebf97a-2f73-4101-a6fa-25ef307904e4 service nova] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:33.565945 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c6015-dd11-4003-9812-10232a61c539 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:33.571414 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc87f5-76fa-480e-aa3b-7e3883cbbc1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:33.572283 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc87f5-76fa-480e-aa3b-7e3883cbbc1a service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:33.572741 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc87f5-76fa-480e-aa3b-7e3883cbbc1a service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:33.573301 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc87f5-76fa-480e-aa3b-7e3883cbbc1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:33.679660 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65fccd1-2f41-4986-ae0a-9e97c8e59c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:33.680288 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65fccd1-2f41-4986-ae0a-9e97c8e59c92 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:33.680565 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65fccd1-2f41-4986-ae0a-9e97c8e59c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:33.680830 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65fccd1-2f41-4986-ae0a-9e97c8e59c92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:33.681479 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65fccd1-2f41-4986-ae0a-9e97c8e59c92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:33.685599 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65fccd1-2f41-4986-ae0a-9e97c8e59c92 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:33.707459 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 829/1632] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:18:33.708726 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65fccd1-2f41-4986-ae0a-9e97c8e59c92 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:33.740686 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fc87f5-76fa-480e-aa3b-7e3883cbbc1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:33.742842 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65fccd1-2f41-4986-ae0a-9e97c8e59c92 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:33.743673 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fc87f5-76fa-480e-aa3b-7e3883cbbc1a service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:33.756312 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0efc99-3e44-4e56-a726-c281a166caeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:33.756948 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0efc99-3e44-4e56-a726-c281a166caeb admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:33.757221 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0efc99-3e44-4e56-a726-c281a166caeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:33.757489 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0efc99-3e44-4e56-a726-c281a166caeb admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:33.758442 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0efc99-3e44-4e56-a726-c281a166caeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:33.760459 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0efc99-3e44-4e56-a726-c281a166caeb admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:33.772251 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 804/1633] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:18:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:18:33.773585 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d224b1e-75a0-4c56-82a8-191a4a5c24ae None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:33.776026 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d224b1e-75a0-4c56-82a8-191a4a5c24ae None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:33.776303 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d224b1e-75a0-4c56-82a8-191a4a5c24ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:33.781173 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 830/1634] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:18:33.781999 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0efc99-3e44-4e56-a726-c281a166caeb admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:33.808911 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0efc99-3e44-4e56-a726-c281a166caeb admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:33.820879 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbc9b1c-ce93-4a65-ba21-5cb3db3a3e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:33.821560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc9b1c-ce93-4a65-ba21-5cb3db3a3e36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:33.824265 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc9b1c-ce93-4a65-ba21-5cb3db3a3e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:33.826559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc9b1c-ce93-4a65-ba21-5cb3db3a3e36 admin admin] PATH_INFO: `/v3/projects/4a3839bf23f645c19a567b642c7e3238/users/1fd72285ddf645049ae486a03a986567/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:33.832377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbc9b1c-ce93-4a65-ba21-5cb3db3a3e36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a3839bf23f645c19a567b642c7e3238, user_id=1fd72285ddf645049ae486a03a986567, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:33.837045 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbc9b1c-ce93-4a65-ba21-5cb3db3a3e36 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:33.842717 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-7d224b1e-75a0-4c56-82a8-191a4a5c24ae None None] MFA Rules not processed for user `5d566ef3f774464189527523acbe908c`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:33.843948 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-7d224b1e-75a0-4c56-82a8-191a4a5c24ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:33.869755 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 831/1635] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:18:33 2022] PUT /identity/v3/projects/4a3839bf23f645c19a567b642c7e3238/users/1fd72285ddf645049ae486a03a986567/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:33.869755 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbc9b1c-ce93-4a65-ba21-5cb3db3a3e36 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:34.009168 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 805/1636] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:33 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:34.011340 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c49ec-6ad9-4daf-ac47-e84e78eb0417 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:34.011771 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c49ec-6ad9-4daf-ac47-e84e78eb0417 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:34.012186 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c49ec-6ad9-4daf-ac47-e84e78eb0417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:34.036294 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbc9b1c-ce93-4a65-ba21-5cb3db3a3e36 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:34.048299 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b156a2-e22d-45f3-afa4-a812797e3ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:34.049422 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b156a2-e22d-45f3-afa4-a812797e3ffe admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:34.049422 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b156a2-e22d-45f3-afa4-a812797e3ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:34.049668 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b156a2-e22d-45f3-afa4-a812797e3ffe admin admin] PATH_INFO: `/v3/projects/382300da8cf0482a96ab096871b75675/users/e7b35249c1544746bd8d4340821cf30b/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:34.054572 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b156a2-e22d-45f3-afa4-a812797e3ffe admin admin] RBAC: Authorizing `identity:create_grant(project_id=382300da8cf0482a96ab096871b75675, user_id=e7b35249c1544746bd8d4340821cf30b, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:34.062943 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b156a2-e22d-45f3-afa4-a812797e3ffe admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:34.085021 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 832/1637] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:18:33 2022] PUT /identity/v3/projects/382300da8cf0482a96ab096871b75675/users/e7b35249c1544746bd8d4340821cf30b/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:34.085631 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b156a2-e22d-45f3-afa4-a812797e3ffe admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:34.096789 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-c05c49ec-6ad9-4daf-ac47-e84e78eb0417 None None] MFA Rules not processed for user `1fd72285ddf645049ae486a03a986567`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:34.098034 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c05c49ec-6ad9-4daf-ac47-e84e78eb0417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:34.179799 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:34.246274 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b156a2-e22d-45f3-afa4-a812797e3ffe admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:34.273996 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b119a7e1-980e-4aeb-a9f9-c91158c79bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:34.274663 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119a7e1-980e-4aeb-a9f9-c91158c79bdd admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:34.274941 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119a7e1-980e-4aeb-a9f9-c91158c79bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:34.275206 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119a7e1-980e-4aeb-a9f9-c91158c79bdd admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:34.275843 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b119a7e1-980e-4aeb-a9f9-c91158c79bdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:34.277754 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b119a7e1-980e-4aeb-a9f9-c91158c79bdd admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:34.285940 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 806/1638] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:34 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:34.287335 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc25a302-fa2e-4d26-b9d8-415bab12bafa None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:34.287615 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc25a302-fa2e-4d26-b9d8-415bab12bafa None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:34.287882 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc25a302-fa2e-4d26-b9d8-415bab12bafa None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:34.289198 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 807/1639] 10.222.0.36 () {54 vars in 912 bytes} [Wed Aug 31 05:18:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:18:34.295138 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5230d8-11a7-4211-b6bf-580cb474e22d None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:34.295411 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5230d8-11a7-4211-b6bf-580cb474e22d None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:34.295670 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5230d8-11a7-4211-b6bf-580cb474e22d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:34.300337 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:34.364046 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 833/1640] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:18:34 2022] POST /identity/v3/projects => generated 403 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:18:34.365387 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c726a-03d3-4c87-add4-fcbf1a690090 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:34.365665 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c726a-03d3-4c87-add4-fcbf1a690090 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:34.365928 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c726a-03d3-4c87-add4-fcbf1a690090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:34.389721 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-1a5230d8-11a7-4211-b6bf-580cb474e22d None None] MFA Rules not processed for user `e7b35249c1544746bd8d4340821cf30b`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:34.390982 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-1a5230d8-11a7-4211-b6bf-580cb474e22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:34.411623 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a88c726a-03d3-4c87-add4-fcbf1a690090 None None] MFA Rules not processed for user `f27665b1574a4db7b7a9771f7d775826`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:34.412920 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a88c726a-03d3-4c87-add4-fcbf1a690090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:34.519504 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 834/1641] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Aug 31 05:18:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:34.523013 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88c726a-03d3-4c87-add4-fcbf1a690090 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:34.539344 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 808/1642] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:34 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:34.540932 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5230d8-11a7-4211-b6bf-580cb474e22d None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:34.689749 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88c726a-03d3-4c87-add4-fcbf1a690090 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:34.698164 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c8d1e4-03e1-4dee-8ee1-09efe4c9ffae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:34.698799 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8d1e4-03e1-4dee-8ee1-09efe4c9ffae service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:34.699074 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8d1e4-03e1-4dee-8ee1-09efe4c9ffae service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:34.699340 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8d1e4-03e1-4dee-8ee1-09efe4c9ffae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:34.748547 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5230d8-11a7-4211-b6bf-580cb474e22d None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:34.763525 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c9cf08-11f9-4399-ba05-3d2af3ffbdd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:34.764226 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9cf08-11f9-4399-ba05-3d2af3ffbdd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:34.764514 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9cf08-11f9-4399-ba05-3d2af3ffbdd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:34.764783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9cf08-11f9-4399-ba05-3d2af3ffbdd7 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:34.767595 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c9cf08-11f9-4399-ba05-3d2af3ffbdd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:34.774329 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c9cf08-11f9-4399-ba05-3d2af3ffbdd7 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:34.835790 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 809/1643] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:18:34 2022] POST /identity/v3/users => generated 308 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:18:34.836885 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c9cf08-11f9-4399-ba05-3d2af3ffbdd7 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:34.861322 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c9cf08-11f9-4399-ba05-3d2af3ffbdd7 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:34.894731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb09e3-4d81-4086-9d4e-d3425c9084b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:34.898475 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb09e3-4d81-4086-9d4e-d3425c9084b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:34.898791 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb09e3-4d81-4086-9d4e-d3425c9084b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:34.899070 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb09e3-4d81-4086-9d4e-d3425c9084b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:34.912209 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c8d1e4-03e1-4dee-8ee1-09efe4c9ffae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:34.912209 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c8d1e4-03e1-4dee-8ee1-09efe4c9ffae service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:34.923081 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 835/1644] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:18:34 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:18:34.925980 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb09e3-4d81-4086-9d4e-d3425c9084b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:34.928973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c8d1e4-03e1-4dee-8ee1-09efe4c9ffae service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:34.944196 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb09e3-4d81-4086-9d4e-d3425c9084b6 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:34.954672 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c8d1e4-03e1-4dee-8ee1-09efe4c9ffae service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:34.964369 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339ba5d-9f17-4a52-a4a3-e961ed93a69c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:34.966616 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339ba5d-9f17-4a52-a4a3-e961ed93a69c service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:34.975623 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 810/1645] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:18:34 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:18:34.978567 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339ba5d-9f17-4a52-a4a3-e961ed93a69c service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:34.978567 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339ba5d-9f17-4a52-a4a3-e961ed93a69c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:34.981147 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb09e3-4d81-4086-9d4e-d3425c9084b6 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:35.000420 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a339ba5d-9f17-4a52-a4a3-e961ed93a69c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:35.000662 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb09e3-4d81-4086-9d4e-d3425c9084b6 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:35.002905 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a339ba5d-9f17-4a52-a4a3-e961ed93a69c service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:35.015981 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a7eac9-91e2-4c57-a9b4-bbc120093403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:35.017719 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7eac9-91e2-4c57-a9b4-bbc120093403 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:35.017719 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7eac9-91e2-4c57-a9b4-bbc120093403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:35.017962 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7eac9-91e2-4c57-a9b4-bbc120093403 admin admin] PATH_INFO: `/v3/projects/7e3c1e9b224e45f0978ba15ea2308094/users/f1e26c38b9fa443fbfc2d251147a4017/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:35.055783 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 836/1646] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:18:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:18:35.095931 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a7eac9-91e2-4c57-a9b4-bbc120093403 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e3c1e9b224e45f0978ba15ea2308094, user_id=f1e26c38b9fa443fbfc2d251147a4017, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:35.098981 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a7eac9-91e2-4c57-a9b4-bbc120093403 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:35.130500 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 811/1647] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:18:34 2022] PUT /identity/v3/projects/7e3c1e9b224e45f0978ba15ea2308094/users/f1e26c38b9fa443fbfc2d251147a4017/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:35.134409 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339ba5d-9f17-4a52-a4a3-e961ed93a69c service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:35.153682 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339ba5d-9f17-4a52-a4a3-e961ed93a69c service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:35.184737 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:35.322316 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f492dc74-12b6-493b-bd26-54932046e305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:35.326216 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f492dc74-12b6-493b-bd26-54932046e305 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:35.327845 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f492dc74-12b6-493b-bd26-54932046e305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:35.331551 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f492dc74-12b6-493b-bd26-54932046e305 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:35.332797 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f492dc74-12b6-493b-bd26-54932046e305 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:35.337242 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f492dc74-12b6-493b-bd26-54932046e305 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:35.381026 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 837/1648] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:18:35 2022] GET /identity/v3/roles => generated 1420 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:18:35.384146 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a7eac9-91e2-4c57-a9b4-bbc120093403 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:35.393260 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f492dc74-12b6-493b-bd26-54932046e305 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:35.408435 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:35.424743 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f492dc74-12b6-493b-bd26-54932046e305 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:35.436057 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a96173a-0d9f-4dff-bac4-320b33ed28d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:35.436939 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96173a-0d9f-4dff-bac4-320b33ed28d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:35.437303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96173a-0d9f-4dff-bac4-320b33ed28d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:35.437684 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96173a-0d9f-4dff-bac4-320b33ed28d4 admin admin] PATH_INFO: `/v3/projects/7e3c1e9b224e45f0978ba15ea2308094/users/f1e26c38b9fa443fbfc2d251147a4017/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:35.442439 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a96173a-0d9f-4dff-bac4-320b33ed28d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e3c1e9b224e45f0978ba15ea2308094, user_id=f1e26c38b9fa443fbfc2d251147a4017, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:35.447804 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a96173a-0d9f-4dff-bac4-320b33ed28d4 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:35.468574 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 838/1649] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:18:35 2022] PUT /identity/v3/projects/7e3c1e9b224e45f0978ba15ea2308094/users/f1e26c38b9fa443fbfc2d251147a4017/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:35.534062 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a7eac9-91e2-4c57-a9b4-bbc120093403 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:35.659040 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a38416-4766-4538-a9f3-fa817118100c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:35.660626 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a38416-4766-4538-a9f3-fa817118100c service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:35.662484 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a38416-4766-4538-a9f3-fa817118100c service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:35.662616 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a38416-4766-4538-a9f3-fa817118100c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:35.794914 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a38416-4766-4538-a9f3-fa817118100c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:35.798647 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a38416-4766-4538-a9f3-fa817118100c service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:35.835264 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 812/1650] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:18:35 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:18:36.158898 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a96173a-0d9f-4dff-bac4-320b33ed28d4 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:36.162773 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a38416-4766-4538-a9f3-fa817118100c service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:36.174445 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a38416-4766-4538-a9f3-fa817118100c service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:36.190848 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755b7d1f-0161-4f52-9d09-7be230e360ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:36.194571 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b7d1f-0161-4f52-9d09-7be230e360ff service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:36.194863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b7d1f-0161-4f52-9d09-7be230e360ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:36.197598 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b7d1f-0161-4f52-9d09-7be230e360ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:36.315747 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a96173a-0d9f-4dff-bac4-320b33ed28d4 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:36.344653 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d7b35d-121c-4806-ac58-02f5a2508c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:36.345882 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d7b35d-121c-4806-ac58-02f5a2508c16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:36.346338 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755b7d1f-0161-4f52-9d09-7be230e360ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:36.346684 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d7b35d-121c-4806-ac58-02f5a2508c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:36.347159 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d7b35d-121c-4806-ac58-02f5a2508c16 admin admin] PATH_INFO: `/v3/users/1f6e9fa5046c4b179b49d3efbe3845ed` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:36.347820 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755b7d1f-0161-4f52-9d09-7be230e360ff service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:36.348872 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d7b35d-121c-4806-ac58-02f5a2508c16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f6e9fa5046c4b179b49d3efbe3845ed)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:36.351237 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d7b35d-121c-4806-ac58-02f5a2508c16 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:36.374805 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 813/1651] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:18:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:18:36.433607 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-04d7b35d-121c-4806-ac58-02f5a2508c16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f6e9fa5046c4b179b49d3efbe3845ed'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:36.441826 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-04d7b35d-121c-4806-ac58-02f5a2508c16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f6e9fa5046c4b179b49d3efbe3845ed'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:36.452546 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-04d7b35d-121c-4806-ac58-02f5a2508c16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f6e9fa5046c4b179b49d3efbe3845ed'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:36.453342 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-04d7b35d-121c-4806-ac58-02f5a2508c16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f6e9fa5046c4b179b49d3efbe3845ed'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:36.467937 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-04d7b35d-121c-4806-ac58-02f5a2508c16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f6e9fa5046c4b179b49d3efbe3845ed'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:36.477552 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 839/1652] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:18:36 2022] DELETE /identity/v3/users/1f6e9fa5046c4b179b49d3efbe3845ed => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:36.481987 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151786da-2178-45e8-818c-064e5c3f6c62 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:36.482306 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151786da-2178-45e8-818c-064e5c3f6c62 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:36.482637 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151786da-2178-45e8-818c-064e5c3f6c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:36.511361 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-151786da-2178-45e8-818c-064e5c3f6c62 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:36.513804 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-151786da-2178-45e8-818c-064e5c3f6c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:36.534909 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:36.676780 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 814/1653] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:36.729023 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d7b35d-121c-4806-ac58-02f5a2508c16 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:36.731405 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-04d7b35d-121c-4806-ac58-02f5a2508c16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:36.777074 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:36.829587 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d7b35d-121c-4806-ac58-02f5a2508c16 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:36.835930 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cbca6c-bb19-4309-913e-603f4f6e6474 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:36.836781 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cbca6c-bb19-4309-913e-603f4f6e6474 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:36.837159 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cbca6c-bb19-4309-913e-603f4f6e6474 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:36.837567 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cbca6c-bb19-4309-913e-603f4f6e6474 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:36.851505 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cbca6c-bb19-4309-913e-603f4f6e6474 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:36.853655 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cbca6c-bb19-4309-913e-603f4f6e6474 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:36.867811 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 840/1654] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:18:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:18:38.245054 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151786da-2178-45e8-818c-064e5c3f6c62 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:38.248471 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-151786da-2178-45e8-818c-064e5c3f6c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:38.276407 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151786da-2178-45e8-818c-064e5c3f6c62 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:38.312457 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9188fe02-67b0-47f1-bc57-1ee949a08259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:38.313163 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188fe02-67b0-47f1-bc57-1ee949a08259 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:38.313470 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188fe02-67b0-47f1-bc57-1ee949a08259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:38.313735 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188fe02-67b0-47f1-bc57-1ee949a08259 admin admin] PATH_INFO: `/v3/projects/ac903c54f50c490cbf4aacf457bed01b` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:38.316120 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9188fe02-67b0-47f1-bc57-1ee949a08259 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac903c54f50c490cbf4aacf457bed01b)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:38.318149 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9188fe02-67b0-47f1-bc57-1ee949a08259 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:38.481983 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9188fe02-67b0-47f1-bc57-1ee949a08259 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac903c54f50c490cbf4aacf457bed01b', 'request_id': 'req-9188fe02-67b0-47f1-bc57-1ee949a08259'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:38.511479 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9188fe02-67b0-47f1-bc57-1ee949a08259 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac903c54f50c490cbf4aacf457bed01b', 'request_id': 'req-9188fe02-67b0-47f1-bc57-1ee949a08259'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:38.536169 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9188fe02-67b0-47f1-bc57-1ee949a08259 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac903c54f50c490cbf4aacf457bed01b', 'request_id': 'req-9188fe02-67b0-47f1-bc57-1ee949a08259'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:38.563908 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9188fe02-67b0-47f1-bc57-1ee949a08259 admin admin] The token cache is being invalidate because project ac903c54f50c490cbf4aacf457bed01b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:18:38.564195 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9188fe02-67b0-47f1-bc57-1ee949a08259 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:38.564863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9188fe02-67b0-47f1-bc57-1ee949a08259 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:38.566748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 815/1655] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:18:38 2022] DELETE /identity/v3/projects/ac903c54f50c490cbf4aacf457bed01b => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:38.571409 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cbca6c-bb19-4309-913e-603f4f6e6474 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:38.603694 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2dd97b-d4c9-4786-95ef-7f1a3c8e0600 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:38.603977 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2dd97b-d4c9-4786-95ef-7f1a3c8e0600 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:38.604244 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2dd97b-d4c9-4786-95ef-7f1a3c8e0600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:38.607062 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cbca6c-bb19-4309-913e-603f4f6e6474 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:38.690775 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-db2dd97b-d4c9-4786-95ef-7f1a3c8e0600 None None] MFA Rules not processed for user `f1e26c38b9fa443fbfc2d251147a4017`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:38.692028 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-db2dd97b-d4c9-4786-95ef-7f1a3c8e0600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:38.802465 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d8dc3-45ea-4456-8d70-154719a06119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:38.803106 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d8dc3-45ea-4456-8d70-154719a06119 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:38.803377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d8dc3-45ea-4456-8d70-154719a06119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:38.803639 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d8dc3-45ea-4456-8d70-154719a06119 admin admin] PATH_INFO: `/v3/users/1fd72285ddf645049ae486a03a986567` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:38.814192 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6d8dc3-45ea-4456-8d70-154719a06119 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fd72285ddf645049ae486a03a986567)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:38.816950 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6d8dc3-45ea-4456-8d70-154719a06119 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:38.913425 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 816/1656] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:38 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:38.913425 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2dd97b-d4c9-4786-95ef-7f1a3c8e0600 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:38.914997 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-db2dd97b-d4c9-4786-95ef-7f1a3c8e0600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:38.937031 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2dd97b-d4c9-4786-95ef-7f1a3c8e0600 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:38.949124 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ee6d8dc3-45ea-4456-8d70-154719a06119 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fd72285ddf645049ae486a03a986567'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:38.962566 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361d8add-3f3c-4cd6-849e-6c80ed297fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:38.964869 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d8add-3f3c-4cd6-849e-6c80ed297fc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:38.965386 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d8add-3f3c-4cd6-849e-6c80ed297fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:38.966204 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d8add-3f3c-4cd6-849e-6c80ed297fc7 admin admin] PATH_INFO: `/v3/users/fd7ec99ba8e0464d98840d5f4a38417e` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:38.966428 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ee6d8dc3-45ea-4456-8d70-154719a06119 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fd72285ddf645049ae486a03a986567'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:38.968764 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361d8add-3f3c-4cd6-849e-6c80ed297fc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd7ec99ba8e0464d98840d5f4a38417e)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:38.971056 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: WARNING py.warnings [None req-361d8add-3f3c-4cd6-849e-6c80ed297fc7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 05:18:38.971056 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: warnings.warn(msg) Aug 31 05:18:38.971056 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: Aug 31 05:18:38.975651 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361d8add-3f3c-4cd6-849e-6c80ed297fc7 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:38.988784 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ee6d8dc3-45ea-4456-8d70-154719a06119 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fd72285ddf645049ae486a03a986567'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:38.989548 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ee6d8dc3-45ea-4456-8d70-154719a06119 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fd72285ddf645049ae486a03a986567'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:39.012448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ee6d8dc3-45ea-4456-8d70-154719a06119 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fd72285ddf645049ae486a03a986567'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:39.025898 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 841/1657] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:18:38 2022] DELETE /identity/v3/users/1fd72285ddf645049ae486a03a986567 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:39.063456 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:39.150379 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-361d8add-3f3c-4cd6-849e-6c80ed297fc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd7ec99ba8e0464d98840d5f4a38417e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:39.180910 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-361d8add-3f3c-4cd6-849e-6c80ed297fc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd7ec99ba8e0464d98840d5f4a38417e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:39.181881 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-361d8add-3f3c-4cd6-849e-6c80ed297fc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd7ec99ba8e0464d98840d5f4a38417e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:39.202152 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-361d8add-3f3c-4cd6-849e-6c80ed297fc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd7ec99ba8e0464d98840d5f4a38417e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:39.202858 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-361d8add-3f3c-4cd6-849e-6c80ed297fc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd7ec99ba8e0464d98840d5f4a38417e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:39.274705 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 817/1658] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:18:38 2022] DELETE /identity/v3/users/fd7ec99ba8e0464d98840d5f4a38417e => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:39.290330 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898f382-c974-41e1-9279-0fb1609532b9 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:39.293453 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898f382-c974-41e1-9279-0fb1609532b9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:39.296083 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898f382-c974-41e1-9279-0fb1609532b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:39.325646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-c898f382-c974-41e1-9279-0fb1609532b9 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:39.327702 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c898f382-c974-41e1-9279-0fb1609532b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:39.408292 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:39.460438 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 842/1659] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:39.499782 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361d8add-3f3c-4cd6-849e-6c80ed297fc7 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:39.504304 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-361d8add-3f3c-4cd6-849e-6c80ed297fc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:39.593478 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361d8add-3f3c-4cd6-849e-6c80ed297fc7 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:39.598156 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c2d68-b462-4126-8064-42f5d0a40bf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:39.598961 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c2d68-b462-4126-8064-42f5d0a40bf2 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:39.599819 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c2d68-b462-4126-8064-42f5d0a40bf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:39.600207 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c2d68-b462-4126-8064-42f5d0a40bf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:39.616322 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4c2d68-b462-4126-8064-42f5d0a40bf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:39.618754 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4c2d68-b462-4126-8064-42f5d0a40bf2 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:39.630839 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 818/1660] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:18:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:18:40.558944 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c898f382-c974-41e1-9279-0fb1609532b9 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:40.560977 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c898f382-c974-41e1-9279-0fb1609532b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:40.576174 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c898f382-c974-41e1-9279-0fb1609532b9 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:40.581762 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb3252c-4d8d-4e18-952a-c2c674a40538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:40.582665 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb3252c-4d8d-4e18-952a-c2c674a40538 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:40.583081 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb3252c-4d8d-4e18-952a-c2c674a40538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:40.583346 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb3252c-4d8d-4e18-952a-c2c674a40538 admin admin] PATH_INFO: `/v3/projects/7c7f893cbdb04887bfc04fcc56da939e` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:40.584561 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb3252c-4d8d-4e18-952a-c2c674a40538 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c7f893cbdb04887bfc04fcc56da939e)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:40.586414 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: WARNING py.warnings [None req-acb3252c-4d8d-4e18-952a-c2c674a40538 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 05:18:40.586414 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: warnings.warn(msg) Aug 31 05:18:40.586414 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: Aug 31 05:18:40.587627 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb3252c-4d8d-4e18-952a-c2c674a40538 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:40.667509 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-acb3252c-4d8d-4e18-952a-c2c674a40538 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c7f893cbdb04887bfc04fcc56da939e', 'request_id': 'req-acb3252c-4d8d-4e18-952a-c2c674a40538'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:40.683612 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-acb3252c-4d8d-4e18-952a-c2c674a40538 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c7f893cbdb04887bfc04fcc56da939e', 'request_id': 'req-acb3252c-4d8d-4e18-952a-c2c674a40538'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:40.709337 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-acb3252c-4d8d-4e18-952a-c2c674a40538 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c7f893cbdb04887bfc04fcc56da939e', 'request_id': 'req-acb3252c-4d8d-4e18-952a-c2c674a40538'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:40.729529 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-acb3252c-4d8d-4e18-952a-c2c674a40538 admin admin] The token cache is being invalidate because project 7c7f893cbdb04887bfc04fcc56da939e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:18:40.731200 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-acb3252c-4d8d-4e18-952a-c2c674a40538 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:40.733219 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-acb3252c-4d8d-4e18-952a-c2c674a40538 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:40.735484 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 843/1661] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:18:40 2022] DELETE /identity/v3/projects/7c7f893cbdb04887bfc04fcc56da939e => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:40.741458 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d818d-6109-4253-ad09-4ec2d1a5975e None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:40.741458 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d818d-6109-4253-ad09-4ec2d1a5975e None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:40.741458 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d818d-6109-4253-ad09-4ec2d1a5975e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:40.777071 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-7e9d818d-6109-4253-ad09-4ec2d1a5975e None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:40.778979 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-7e9d818d-6109-4253-ad09-4ec2d1a5975e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:40.868842 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b7daa-c4dd-4153-b93b-0407e6c72d85 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:40.869099 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b7daa-c4dd-4153-b93b-0407e6c72d85 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:40.869267 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b7daa-c4dd-4153-b93b-0407e6c72d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:40.905029 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-df8b7daa-c4dd-4153-b93b-0407e6c72d85 None None] MFA Rules not processed for user `69e6e2487fce4c03a8770443791c5b8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:40.907369 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-df8b7daa-c4dd-4153-b93b-0407e6c72d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:40.917108 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 819/1662] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:40.925670 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ffe5a2-1dde-43c1-9e2e-24cae06cc1ee None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:40.925962 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ffe5a2-1dde-43c1-9e2e-24cae06cc1ee None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:40.926280 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ffe5a2-1dde-43c1-9e2e-24cae06cc1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:40.978389 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-83ffe5a2-1dde-43c1-9e2e-24cae06cc1ee None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:40.982568 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-83ffe5a2-1dde-43c1-9e2e-24cae06cc1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:41.006863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 820/1663] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:41.011437 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ffe5a2-1dde-43c1-9e2e-24cae06cc1ee None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:41.025609 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ffe5a2-1dde-43c1-9e2e-24cae06cc1ee None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:41.051302 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce6568-421e-4793-a819-50b3600a2866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:41.052726 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce6568-421e-4793-a819-50b3600a2866 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:41.053179 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce6568-421e-4793-a819-50b3600a2866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:41.053657 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce6568-421e-4793-a819-50b3600a2866 admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:41.054648 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ce6568-421e-4793-a819-50b3600a2866 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:18:41.055097 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ce6568-421e-4793-a819-50b3600a2866 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:41.057479 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ce6568-421e-4793-a819-50b3600a2866 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:41.086259 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 821/1664] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:18:41.086449 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 844/1665] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:41.095932 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e911e2b-71b2-4adb-b5af-bcef2fae2986 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:41.096067 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce6568-421e-4793-a819-50b3600a2866 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:41.096214 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e911e2b-71b2-4adb-b5af-bcef2fae2986 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:41.096485 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e911e2b-71b2-4adb-b5af-bcef2fae2986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:41.134264 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce6568-421e-4793-a819-50b3600a2866 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:41.135610 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-3e911e2b-71b2-4adb-b5af-bcef2fae2986 None None] MFA Rules not processed for user `69e6e2487fce4c03a8770443791c5b8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:41.136851 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3e911e2b-71b2-4adb-b5af-bcef2fae2986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:41.144408 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6def7e-56d5-495c-9ba6-3230de2d35c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:41.146149 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6def7e-56d5-495c-9ba6-3230de2d35c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:41.146572 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6def7e-56d5-495c-9ba6-3230de2d35c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:41.146881 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6def7e-56d5-495c-9ba6-3230de2d35c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:41.148376 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6def7e-56d5-495c-9ba6-3230de2d35c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:41.151219 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6def7e-56d5-495c-9ba6-3230de2d35c9 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:41.164820 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 845/1666] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:41.170484 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d985cb5-7736-4b95-b40f-173ac01dc488 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:41.170855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d985cb5-7736-4b95-b40f-173ac01dc488 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:41.171127 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d985cb5-7736-4b95-b40f-173ac01dc488 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:41.174079 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 846/1667] 10.222.0.36 () {54 vars in 921 bytes} [Wed Aug 31 05:18:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:18:41.177617 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d985cb5-7736-4b95-b40f-173ac01dc488 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:41.186349 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1d985cb5-7736-4b95-b40f-173ac01dc488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:41.194303 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:41.212555 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 822/1668] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:18:41 2022] POST /identity/v3/projects => generated 365 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:18:41.216089 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6def7e-56d5-495c-9ba6-3230de2d35c9 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:41.308537 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:41.400406 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d985cb5-7736-4b95-b40f-173ac01dc488 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:41.401367 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6def7e-56d5-495c-9ba6-3230de2d35c9 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:41.407796 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187fd7b-db89-4359-9c10-6b4e7f631289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:41.408603 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187fd7b-db89-4359-9c10-6b4e7f631289 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:41.408982 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187fd7b-db89-4359-9c10-6b4e7f631289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:41.409328 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187fd7b-db89-4359-9c10-6b4e7f631289 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:41.410190 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4187fd7b-db89-4359-9c10-6b4e7f631289 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:41.412629 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4187fd7b-db89-4359-9c10-6b4e7f631289 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:41.453197 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 823/1669] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:18:41 2022] POST /identity/v3/users => generated 283 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:18:41.454729 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44e989-feb6-4af5-9823-7b2b1818449d None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:41.457553 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44e989-feb6-4af5-9823-7b2b1818449d None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:41.457858 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44e989-feb6-4af5-9823-7b2b1818449d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:41.526973 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-cb44e989-feb6-4af5-9823-7b2b1818449d None None] MFA Rules not processed for user `bd4387035099416789009f0ecb8ad282`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:41.528643 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-cb44e989-feb6-4af5-9823-7b2b1818449d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:41.557750 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9b2b79-8f00-423a-9180-a72bdf2a6c61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:41.558629 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b2b79-8f00-423a-9180-a72bdf2a6c61 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:41.560817 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b2b79-8f00-423a-9180-a72bdf2a6c61 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:41.561420 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b2b79-8f00-423a-9180-a72bdf2a6c61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:41.668392 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9b2b79-8f00-423a-9180-a72bdf2a6c61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:41.671037 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9b2b79-8f00-423a-9180-a72bdf2a6c61 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:41.686914 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 847/1670] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:18:41 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:18:41.687934 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9b2b79-8f00-423a-9180-a72bdf2a6c61 service nova] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:41.689655 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5b9b2b79-8f00-423a-9180-a72bdf2a6c61 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:41.715707 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9b2b79-8f00-423a-9180-a72bdf2a6c61 service nova] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:41.736793 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8241e370-c272-4d70-8004-2f3f1db81c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:41.737507 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241e370-c272-4d70-8004-2f3f1db81c76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:41.737787 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241e370-c272-4d70-8004-2f3f1db81c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:41.738102 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241e370-c272-4d70-8004-2f3f1db81c76 admin admin] PATH_INFO: `/v3/projects/4a3839bf23f645c19a567b642c7e3238` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:41.740014 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 824/1671] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Aug 31 05:18:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:41.740599 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8241e370-c272-4d70-8004-2f3f1db81c76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a3839bf23f645c19a567b642c7e3238)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:41.741307 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44e989-feb6-4af5-9823-7b2b1818449d None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:41.742599 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8241e370-c272-4d70-8004-2f3f1db81c76 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:41.773986 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44e989-feb6-4af5-9823-7b2b1818449d None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:41.790923 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7807d9f9-23de-4123-ade3-1ca3ec580325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:41.792897 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807d9f9-23de-4123-ade3-1ca3ec580325 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:41.794149 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807d9f9-23de-4123-ade3-1ca3ec580325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:41.795475 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807d9f9-23de-4123-ade3-1ca3ec580325 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:41.796957 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7807d9f9-23de-4123-ade3-1ca3ec580325 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:41.799876 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7807d9f9-23de-4123-ade3-1ca3ec580325 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:41.836177 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 825/1672] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:18:41 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:18:41.837463 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7807d9f9-23de-4123-ade3-1ca3ec580325 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:41.863711 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7807d9f9-23de-4123-ade3-1ca3ec580325 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:41.866666 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8241e370-c272-4d70-8004-2f3f1db81c76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a3839bf23f645c19a567b642c7e3238', 'request_id': 'req-8241e370-c272-4d70-8004-2f3f1db81c76'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:41.886141 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8241e370-c272-4d70-8004-2f3f1db81c76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a3839bf23f645c19a567b642c7e3238', 'request_id': 'req-8241e370-c272-4d70-8004-2f3f1db81c76'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:41.906467 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8241e370-c272-4d70-8004-2f3f1db81c76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a3839bf23f645c19a567b642c7e3238', 'request_id': 'req-8241e370-c272-4d70-8004-2f3f1db81c76'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:41.929530 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8241e370-c272-4d70-8004-2f3f1db81c76 admin admin] The token cache is being invalidate because project 4a3839bf23f645c19a567b642c7e3238 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:18:41.929821 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8241e370-c272-4d70-8004-2f3f1db81c76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:41.936273 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8241e370-c272-4d70-8004-2f3f1db81c76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:41.938197 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 848/1673] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:18:41 2022] DELETE /identity/v3/projects/4a3839bf23f645c19a567b642c7e3238 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:41.938944 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8241e370-c272-4d70-8004-2f3f1db81c76 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:41.944613 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8241e370-c272-4d70-8004-2f3f1db81c76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:41.957073 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:42.064721 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2a2890-95b3-45c0-a5cb-db95d5b5e0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:42.065521 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a2890-95b3-45c0-a5cb-db95d5b5e0e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:42.065906 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a2890-95b3-45c0-a5cb-db95d5b5e0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:42.066313 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a2890-95b3-45c0-a5cb-db95d5b5e0e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:42.067170 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a2890-95b3-45c0-a5cb-db95d5b5e0e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:42.073856 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a2890-95b3-45c0-a5cb-db95d5b5e0e7 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:42.101727 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 826/1674] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:18:41 2022] GET /identity/v3/roles => generated 1420 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:18:42.103394 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968faefc-ca24-49a0-82ad-95c570f16f9f None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:42.108380 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968faefc-ca24-49a0-82ad-95c570f16f9f None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:42.108825 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968faefc-ca24-49a0-82ad-95c570f16f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:42.125093 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8241e370-c272-4d70-8004-2f3f1db81c76 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:42.137238 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e1a6e-ac8f-40c4-8012-923567a58ff0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd4387035099416789009f0ecb8ad282', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:42.138096 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e1a6e-ac8f-40c4-8012-923567a58ff0 service placement] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:42.139025 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e1a6e-ac8f-40c4-8012-923567a58ff0 service placement] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:42.139313 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e1a6e-ac8f-40c4-8012-923567a58ff0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:42.144059 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-052e1a6e-ac8f-40c4-8012-923567a58ff0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:42.158052 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-968faefc-ca24-49a0-82ad-95c570f16f9f None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:42.163256 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-968faefc-ca24-49a0-82ad-95c570f16f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:42.171920 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052e1a6e-ac8f-40c4-8012-923567a58ff0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:42.176055 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052e1a6e-ac8f-40c4-8012-923567a58ff0 service placement] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:42.196243 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 827/1675] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:42.204889 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968faefc-ca24-49a0-82ad-95c570f16f9f None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:42.211160 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-968faefc-ca24-49a0-82ad-95c570f16f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:42.218642 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 849/1676] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:18:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:18:42.227065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096caae1-c462-4fdc-aa57-21b900bf56c8 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:42.234636 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096caae1-c462-4fdc-aa57-21b900bf56c8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:42.236104 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096caae1-c462-4fdc-aa57-21b900bf56c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:42.246404 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:42.260147 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968faefc-ca24-49a0-82ad-95c570f16f9f None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:42.277511 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1ce1b-41fc-48c7-8f7e-0b1a7b7fa847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:42.281018 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1ce1b-41fc-48c7-8f7e-0b1a7b7fa847 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:42.281145 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1ce1b-41fc-48c7-8f7e-0b1a7b7fa847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:42.292735 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1ce1b-41fc-48c7-8f7e-0b1a7b7fa847 admin admin] PATH_INFO: `/v3/projects/b92519408725413db28b3d593ff63a1f/users/8e174a6bc62f4cf395d50a93725094e5/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:42.327810 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-096caae1-c462-4fdc-aa57-21b900bf56c8 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:42.332136 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-096caae1-c462-4fdc-aa57-21b900bf56c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:42.368345 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd1ce1b-41fc-48c7-8f7e-0b1a7b7fa847 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b92519408725413db28b3d593ff63a1f, user_id=8e174a6bc62f4cf395d50a93725094e5, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:42.371872 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd1ce1b-41fc-48c7-8f7e-0b1a7b7fa847 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:42.375680 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 850/1677] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:42.381427 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096caae1-c462-4fdc-aa57-21b900bf56c8 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:42.418473 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096caae1-c462-4fdc-aa57-21b900bf56c8 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:42.441347 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 828/1678] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:18:42 2022] PUT /identity/v3/projects/b92519408725413db28b3d593ff63a1f/users/8e174a6bc62f4cf395d50a93725094e5/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:42.451885 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964a065-b8f0-469f-973d-3edc7af57186 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:42.458187 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964a065-b8f0-469f-973d-3edc7af57186 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:42.458571 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964a065-b8f0-469f-973d-3edc7af57186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:42.560934 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-f964a065-b8f0-469f-973d-3edc7af57186 None None] MFA Rules not processed for user `8e174a6bc62f4cf395d50a93725094e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:42.567491 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-f964a065-b8f0-469f-973d-3edc7af57186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:42.657866 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0313ef8-3edf-4ca7-8e84-6a4c37ef560e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:42.659457 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0313ef8-3edf-4ca7-8e84-6a4c37ef560e admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:42.659457 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0313ef8-3edf-4ca7-8e84-6a4c37ef560e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:42.659675 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0313ef8-3edf-4ca7-8e84-6a4c37ef560e admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:42.660312 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0313ef8-3edf-4ca7-8e84-6a4c37ef560e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:18:42.660780 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0313ef8-3edf-4ca7-8e84-6a4c37ef560e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:42.662714 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0313ef8-3edf-4ca7-8e84-6a4c37ef560e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:42.695541 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 851/1679] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:18:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:18:42.696577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0313ef8-3edf-4ca7-8e84-6a4c37ef560e admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:42.724717 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0313ef8-3edf-4ca7-8e84-6a4c37ef560e admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:42.738682 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52758fe3-d8ef-4698-817e-979c56c8089a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:42.740162 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52758fe3-d8ef-4698-817e-979c56c8089a admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:42.741888 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52758fe3-d8ef-4698-817e-979c56c8089a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:42.742366 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52758fe3-d8ef-4698-817e-979c56c8089a admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:42.744276 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52758fe3-d8ef-4698-817e-979c56c8089a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:42.748971 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 829/1680] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:42.751652 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52758fe3-d8ef-4698-817e-979c56c8089a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:42.762530 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f964a065-b8f0-469f-973d-3edc7af57186 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:42.764106 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-f964a065-b8f0-469f-973d-3edc7af57186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:42.813055 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 852/1681] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:18:42 2022] POST /identity/v3/projects => generated 387 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:18:42.816857 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52758fe3-d8ef-4698-817e-979c56c8089a admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:42.906974 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f964a065-b8f0-469f-973d-3edc7af57186 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:42.950397 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:42.958494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52758fe3-d8ef-4698-817e-979c56c8089a admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:42.971939 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0738920e-797f-495b-95ad-a14e7d56befd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:42.972897 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738920e-797f-495b-95ad-a14e7d56befd admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:42.973369 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738920e-797f-495b-95ad-a14e7d56befd admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:42.973818 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738920e-797f-495b-95ad-a14e7d56befd admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:42.974802 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0738920e-797f-495b-95ad-a14e7d56befd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:42.976940 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0738920e-797f-495b-95ad-a14e7d56befd admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:43.022740 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 853/1682] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:18:42 2022] POST /identity/v3/users => generated 294 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:18:43.026848 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0738920e-797f-495b-95ad-a14e7d56befd admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:43.030766 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126de4cc-d4e2-4ee7-993b-75a625d313fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:43.031571 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126de4cc-d4e2-4ee7-993b-75a625d313fd service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:43.033297 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126de4cc-d4e2-4ee7-993b-75a625d313fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:43.033588 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126de4cc-d4e2-4ee7-993b-75a625d313fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:43.049095 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0738920e-797f-495b-95ad-a14e7d56befd admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:43.064517 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58199579-b401-4913-b085-a9306db11af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:43.065774 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58199579-b401-4913-b085-a9306db11af3 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:43.066243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58199579-b401-4913-b085-a9306db11af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:43.067266 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58199579-b401-4913-b085-a9306db11af3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:43.067739 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58199579-b401-4913-b085-a9306db11af3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:43.069575 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58199579-b401-4913-b085-a9306db11af3 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:43.095452 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 854/1683] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:18:43 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:18:43.099627 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58199579-b401-4913-b085-a9306db11af3 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:43.121971 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58199579-b401-4913-b085-a9306db11af3 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:43.128897 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:43.142869 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02fb40e-bc7f-45e5-938e-cf9551613ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:43.145776 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02fb40e-bc7f-45e5-938e-cf9551613ac2 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:43.146399 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02fb40e-bc7f-45e5-938e-cf9551613ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:43.147833 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02fb40e-bc7f-45e5-938e-cf9551613ac2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:43.150758 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02fb40e-bc7f-45e5-938e-cf9551613ac2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:43.160966 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02fb40e-bc7f-45e5-938e-cf9551613ac2 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:43.171013 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126de4cc-d4e2-4ee7-993b-75a625d313fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:43.173123 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126de4cc-d4e2-4ee7-993b-75a625d313fd service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:43.196033 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 855/1684] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:18:43 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:18:43.208354 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2efad60-41a6-427d-991f-c4442748e86e None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:43.208648 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2efad60-41a6-427d-991f-c4442748e86e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:43.211336 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 830/1685] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:18:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:18:43.212521 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126de4cc-d4e2-4ee7-993b-75a625d313fd service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:43.214486 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2efad60-41a6-427d-991f-c4442748e86e None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:43.219339 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 856/1686] 10.222.0.36 () {54 vars in 912 bytes} [Wed Aug 31 05:18:43 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:18:43.224999 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e33a17-d4f3-412a-85b3-448cb17b61a5 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:43.225418 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e33a17-d4f3-412a-85b3-448cb17b61a5 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:43.225785 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e33a17-d4f3-412a-85b3-448cb17b61a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:43.232212 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126de4cc-d4e2-4ee7-993b-75a625d313fd service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:43.239796 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02851780-5919-4216-96c6-b4ada3e1f0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:43.240493 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02851780-5919-4216-96c6-b4ada3e1f0d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:43.240977 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02851780-5919-4216-96c6-b4ada3e1f0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:43.241255 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02851780-5919-4216-96c6-b4ada3e1f0d2 admin admin] PATH_INFO: `/v3/projects/d2582c3ad1214f6cb9fdffeabd950472/users/879bcd89211a432ab36bd5f71b54e8a1/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:43.283112 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02851780-5919-4216-96c6-b4ada3e1f0d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2582c3ad1214f6cb9fdffeabd950472, user_id=879bcd89211a432ab36bd5f71b54e8a1, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:43.285861 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02851780-5919-4216-96c6-b4ada3e1f0d2 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:43.308647 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 831/1687] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:18:43 2022] PUT /identity/v3/projects/d2582c3ad1214f6cb9fdffeabd950472/users/879bcd89211a432ab36bd5f71b54e8a1/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:43.321334 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36dca2e-501d-471a-863c-b87f57c5fec7 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:43.321642 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36dca2e-501d-471a-863c-b87f57c5fec7 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:43.322031 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36dca2e-501d-471a-863c-b87f57c5fec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:43.326642 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-55e33a17-d4f3-412a-85b3-448cb17b61a5 None None] MFA Rules not processed for user `090e1409506542f6831e1f65d44aae74`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:43.330144 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-55e33a17-d4f3-412a-85b3-448cb17b61a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:43.385211 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-b36dca2e-501d-471a-863c-b87f57c5fec7 None None] MFA Rules not processed for user `879bcd89211a432ab36bd5f71b54e8a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:43.386512 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b36dca2e-501d-471a-863c-b87f57c5fec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:43.520093 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 857/1688] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Aug 31 05:18:43 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:18:43.545123 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 832/1689] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:43 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:43.545775 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e33a17-d4f3-412a-85b3-448cb17b61a5 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:43.690644 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e33a17-d4f3-412a-85b3-448cb17b61a5 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:43.704378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ca14db-36d9-4e36-b2c9-95a1d136593c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:43.705011 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca14db-36d9-4e36-b2c9-95a1d136593c admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:43.705292 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca14db-36d9-4e36-b2c9-95a1d136593c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:43.705554 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca14db-36d9-4e36-b2c9-95a1d136593c admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:43.706278 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ca14db-36d9-4e36-b2c9-95a1d136593c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:43.711070 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ca14db-36d9-4e36-b2c9-95a1d136593c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:43.743550 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36dca2e-501d-471a-863c-b87f57c5fec7 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:43.747217 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b36dca2e-501d-471a-863c-b87f57c5fec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:43.777922 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36dca2e-501d-471a-863c-b87f57c5fec7 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:43.787657 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 858/1690] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:18:43 2022] POST /identity/v3/projects => generated 387 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:18:43.788596 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ca14db-36d9-4e36-b2c9-95a1d136593c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:43.793255 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a8ca14db-36d9-4e36-b2c9-95a1d136593c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:43.801500 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10879d83-2ac3-444d-a10f-96bb10ef82b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:43.803413 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10879d83-2ac3-444d-a10f-96bb10ef82b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:43.804158 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10879d83-2ac3-444d-a10f-96bb10ef82b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:43.806029 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10879d83-2ac3-444d-a10f-96bb10ef82b1 admin admin] PATH_INFO: `/v3/users/5d566ef3f774464189527523acbe908c` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:43.810628 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10879d83-2ac3-444d-a10f-96bb10ef82b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d566ef3f774464189527523acbe908c)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:43.815659 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10879d83-2ac3-444d-a10f-96bb10ef82b1 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:43.925182 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:43.947654 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ca14db-36d9-4e36-b2c9-95a1d136593c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:43.952685 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e4067-9dff-4e1a-8875-a8d4f7f67ff3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:43.953304 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e4067-9dff-4e1a-8875-a8d4f7f67ff3 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:43.953705 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e4067-9dff-4e1a-8875-a8d4f7f67ff3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:43.953969 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e4067-9dff-4e1a-8875-a8d4f7f67ff3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:43.967762 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-10879d83-2ac3-444d-a10f-96bb10ef82b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d566ef3f774464189527523acbe908c'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:43.986291 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918e4067-9dff-4e1a-8875-a8d4f7f67ff3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:43.988957 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918e4067-9dff-4e1a-8875-a8d4f7f67ff3 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:43.990839 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-10879d83-2ac3-444d-a10f-96bb10ef82b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d566ef3f774464189527523acbe908c'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:43.991596 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-10879d83-2ac3-444d-a10f-96bb10ef82b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d566ef3f774464189527523acbe908c'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:44.015123 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-10879d83-2ac3-444d-a10f-96bb10ef82b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d566ef3f774464189527523acbe908c'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:44.017995 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-10879d83-2ac3-444d-a10f-96bb10ef82b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d566ef3f774464189527523acbe908c'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:44.022917 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 859/1691] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:18:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:18:44.031276 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e4067-9dff-4e1a-8875-a8d4f7f67ff3 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:44.098580 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 833/1692] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:18:43 2022] DELETE /identity/v3/users/5d566ef3f774464189527523acbe908c => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:18:44.230749 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e4067-9dff-4e1a-8875-a8d4f7f67ff3 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:44.339863 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3a7467-79a1-4817-8d8c-5e375f178b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:44.341019 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a7467-79a1-4817-8d8c-5e375f178b1c admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:44.341380 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a7467-79a1-4817-8d8c-5e375f178b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:44.341719 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a7467-79a1-4817-8d8c-5e375f178b1c admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:44.342622 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3a7467-79a1-4817-8d8c-5e375f178b1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:44.348575 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3a7467-79a1-4817-8d8c-5e375f178b1c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:44.378598 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 860/1693] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:18:44 2022] POST /identity/v3/users => generated 300 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:18:44.382457 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10879d83-2ac3-444d-a10f-96bb10ef82b1 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:44.384225 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-10879d83-2ac3-444d-a10f-96bb10ef82b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:44.410711 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10879d83-2ac3-444d-a10f-96bb10ef82b1 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:44.432365 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeef985-ee86-4d63-88c1-88c481614a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:44.432638 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeef985-ee86-4d63-88c1-88c481614a6b admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:44.432741 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeef985-ee86-4d63-88c1-88c481614a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:44.432975 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeef985-ee86-4d63-88c1-88c481614a6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:44.433628 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeef985-ee86-4d63-88c1-88c481614a6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:44.435361 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeef985-ee86-4d63-88c1-88c481614a6b admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:44.451737 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070355c5-c4eb-455b-a6ee-5763c399e8aa None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:44.452326 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070355c5-c4eb-455b-a6ee-5763c399e8aa None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:44.453440 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070355c5-c4eb-455b-a6ee-5763c399e8aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:44.457086 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 834/1694] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:18:44.461891 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeef985-ee86-4d63-88c1-88c481614a6b admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:44.488585 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeef985-ee86-4d63-88c1-88c481614a6b admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:44.493959 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2577e7a-99e2-44ad-bfc1-e06f82ea1901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:44.494681 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2577e7a-99e2-44ad-bfc1-e06f82ea1901 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:44.494958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2577e7a-99e2-44ad-bfc1-e06f82ea1901 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:44.495226 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2577e7a-99e2-44ad-bfc1-e06f82ea1901 admin admin] PATH_INFO: `/v3/projects/2600bf39f9e04b11a58106167c541350/users/f2be66fc9b3146b6bfecbcc99f60969f/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:44.497647 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:44.528597 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-070355c5-c4eb-455b-a6ee-5763c399e8aa None None] MFA Rules not processed for user `4b07cff5bc3a4e68b62c4c5d3e798732`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:44.530792 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-070355c5-c4eb-455b-a6ee-5763c399e8aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:44.545731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2577e7a-99e2-44ad-bfc1-e06f82ea1901 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2600bf39f9e04b11a58106167c541350, user_id=f2be66fc9b3146b6bfecbcc99f60969f, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:44.549683 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2577e7a-99e2-44ad-bfc1-e06f82ea1901 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:44.570890 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 835/1695] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:18:44 2022] PUT /identity/v3/projects/2600bf39f9e04b11a58106167c541350/users/f2be66fc9b3146b6bfecbcc99f60969f/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:44.576363 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2577e7a-99e2-44ad-bfc1-e06f82ea1901 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:44.697227 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 861/1696] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:44.709416 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070355c5-c4eb-455b-a6ee-5763c399e8aa None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:44.711259 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-070355c5-c4eb-455b-a6ee-5763c399e8aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:44.733424 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2577e7a-99e2-44ad-bfc1-e06f82ea1901 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:44.746229 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bea058-2cce-46ad-8483-8717c21cb2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:44.747117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bea058-2cce-46ad-8483-8717c21cb2c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:44.747752 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bea058-2cce-46ad-8483-8717c21cb2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:44.748229 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bea058-2cce-46ad-8483-8717c21cb2c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:44.751612 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bea058-2cce-46ad-8483-8717c21cb2c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:44.752574 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:44.754302 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bea058-2cce-46ad-8483-8717c21cb2c5 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:44.779968 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 836/1697] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:18:44.793890 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bea058-2cce-46ad-8483-8717c21cb2c5 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:44.805245 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bea058-2cce-46ad-8483-8717c21cb2c5 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:44.810719 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a76c40-3681-4fd0-b9bf-2a54ec6649e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:44.811774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a76c40-3681-4fd0-b9bf-2a54ec6649e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:44.812250 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a76c40-3681-4fd0-b9bf-2a54ec6649e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:44.812714 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a76c40-3681-4fd0-b9bf-2a54ec6649e5 admin admin] PATH_INFO: `/v3/projects/2600bf39f9e04b11a58106167c541350/users/f2be66fc9b3146b6bfecbcc99f60969f/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:44.817654 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a76c40-3681-4fd0-b9bf-2a54ec6649e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2600bf39f9e04b11a58106167c541350, user_id=f2be66fc9b3146b6bfecbcc99f60969f, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:44.832824 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a76c40-3681-4fd0-b9bf-2a54ec6649e5 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:44.858056 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 837/1698] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:18:44 2022] PUT /identity/v3/projects/2600bf39f9e04b11a58106167c541350/users/f2be66fc9b3146b6bfecbcc99f60969f/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:44.877290 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566552e4-4855-4e7b-9d87-65959b50c521 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:44.877576 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566552e4-4855-4e7b-9d87-65959b50c521 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:44.877848 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566552e4-4855-4e7b-9d87-65959b50c521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:44.969680 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070355c5-c4eb-455b-a6ee-5763c399e8aa None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:45.013697 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-566552e4-4855-4e7b-9d87-65959b50c521 None None] MFA Rules not processed for user `f2be66fc9b3146b6bfecbcc99f60969f`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:45.016414 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-566552e4-4855-4e7b-9d87-65959b50c521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:45.188753 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 838/1699] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:44 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:45.189548 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566552e4-4855-4e7b-9d87-65959b50c521 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:45.190984 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-566552e4-4855-4e7b-9d87-65959b50c521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:45.211779 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63da7c-1e63-48f3-88a6-b27850c59c6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:45.212650 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63da7c-1e63-48f3-88a6-b27850c59c6f service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:45.213052 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63da7c-1e63-48f3-88a6-b27850c59c6f service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:45.213326 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63da7c-1e63-48f3-88a6-b27850c59c6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:45.327222 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb63da7c-1e63-48f3-88a6-b27850c59c6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:45.332303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb63da7c-1e63-48f3-88a6-b27850c59c6f service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:45.346093 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:45.351864 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 862/1700] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:18:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 643 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:18:45.353108 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63da7c-1e63-48f3-88a6-b27850c59c6f service nova] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:45.354251 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-eb63da7c-1e63-48f3-88a6-b27850c59c6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:45.390945 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566552e4-4855-4e7b-9d87-65959b50c521 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:45.397475 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd0df8-7aa3-41c9-bec6-e8f9591bc545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:45.398400 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd0df8-7aa3-41c9-bec6-e8f9591bc545 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:45.398941 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd0df8-7aa3-41c9-bec6-e8f9591bc545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:45.399396 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd0df8-7aa3-41c9-bec6-e8f9591bc545 admin admin] PATH_INFO: `/v3/projects/e5368d3255474cd08e31d3c2bd952cc1` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:45.400731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dd0df8-7aa3-41c9-bec6-e8f9591bc545 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5368d3255474cd08e31d3c2bd952cc1)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:45.403717 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dd0df8-7aa3-41c9-bec6-e8f9591bc545 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:45.489829 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63da7c-1e63-48f3-88a6-b27850c59c6f service nova] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:45.502429 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70538242-e53c-4a20-ac58-75fac9d38686 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:45.503305 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70538242-e53c-4a20-ac58-75fac9d38686 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:45.503817 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70538242-e53c-4a20-ac58-75fac9d38686 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:45.504211 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70538242-e53c-4a20-ac58-75fac9d38686 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:45.518448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:45.528209 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70538242-e53c-4a20-ac58-75fac9d38686 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:45.531259 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70538242-e53c-4a20-ac58-75fac9d38686 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:45.577573 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 863/1701] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:18:45 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:18:45.583001 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70538242-e53c-4a20-ac58-75fac9d38686 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:45.585623 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-70538242-e53c-4a20-ac58-75fac9d38686 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:45.597388 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-08dd0df8-7aa3-41c9-bec6-e8f9591bc545 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5368d3255474cd08e31d3c2bd952cc1', 'request_id': 'req-08dd0df8-7aa3-41c9-bec6-e8f9591bc545'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:45.636275 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-08dd0df8-7aa3-41c9-bec6-e8f9591bc545 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5368d3255474cd08e31d3c2bd952cc1', 'request_id': 'req-08dd0df8-7aa3-41c9-bec6-e8f9591bc545'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:45.674980 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-08dd0df8-7aa3-41c9-bec6-e8f9591bc545 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5368d3255474cd08e31d3c2bd952cc1', 'request_id': 'req-08dd0df8-7aa3-41c9-bec6-e8f9591bc545'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:45.757485 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-08dd0df8-7aa3-41c9-bec6-e8f9591bc545 admin admin] The token cache is being invalidate because project e5368d3255474cd08e31d3c2bd952cc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:18:45.762950 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-08dd0df8-7aa3-41c9-bec6-e8f9591bc545 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:45.763932 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-08dd0df8-7aa3-41c9-bec6-e8f9591bc545 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:45.774776 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 839/1702] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:18:45 2022] DELETE /identity/v3/projects/e5368d3255474cd08e31d3c2bd952cc1 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:45.776182 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd0df8-7aa3-41c9-bec6-e8f9591bc545 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:45.789176 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-08dd0df8-7aa3-41c9-bec6-e8f9591bc545 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:45.807101 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70538242-e53c-4a20-ac58-75fac9d38686 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:46.018502 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c6cbdd-c588-4730-a06a-40579ea2d496 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd4387035099416789009f0ecb8ad282', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:46.018502 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6cbdd-c588-4730-a06a-40579ea2d496 service placement] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:46.018502 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6cbdd-c588-4730-a06a-40579ea2d496 service placement] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:46.018502 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6cbdd-c588-4730-a06a-40579ea2d496 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:46.019010 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c0c6cbdd-c588-4730-a06a-40579ea2d496 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:46.026981 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c6cbdd-c588-4730-a06a-40579ea2d496 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:46.029039 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c6cbdd-c588-4730-a06a-40579ea2d496 service placement] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:46.050923 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd0df8-7aa3-41c9-bec6-e8f9591bc545 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:46.054124 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 864/1703] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:18:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:18:46.062862 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d34f8d0-288d-4a84-849e-0f71d9a37b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:46.063617 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34f8d0-288d-4a84-849e-0f71d9a37b47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:46.064225 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34f8d0-288d-4a84-849e-0f71d9a37b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:46.064740 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34f8d0-288d-4a84-849e-0f71d9a37b47 admin admin] PATH_INFO: `/v3/users/879bcd89211a432ab36bd5f71b54e8a1` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:46.065491 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c6cbdd-c588-4730-a06a-40579ea2d496 service placement] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:46.074149 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c0c6cbdd-c588-4730-a06a-40579ea2d496 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:46.083643 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d34f8d0-288d-4a84-849e-0f71d9a37b47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=879bcd89211a432ab36bd5f71b54e8a1)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:46.100659 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d34f8d0-288d-4a84-849e-0f71d9a37b47 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:46.119389 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c6cbdd-c588-4730-a06a-40579ea2d496 service placement] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:46.139841 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d4491-f85c-4293-a904-a9bd30023b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:46.142362 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d4491-f85c-4293-a904-a9bd30023b4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:46.143295 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d4491-f85c-4293-a904-a9bd30023b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:46.144476 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d4491-f85c-4293-a904-a9bd30023b4b admin admin] PATH_INFO: `/v3/users/f1e26c38b9fa443fbfc2d251147a4017` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:46.151549 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979d4491-f85c-4293-a904-a9bd30023b4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1e26c38b9fa443fbfc2d251147a4017)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:46.160833 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979d4491-f85c-4293-a904-a9bd30023b4b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:46.290267 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-979d4491-f85c-4293-a904-a9bd30023b4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1e26c38b9fa443fbfc2d251147a4017'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:46.308714 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-979d4491-f85c-4293-a904-a9bd30023b4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1e26c38b9fa443fbfc2d251147a4017'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:46.320607 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-5d34f8d0-288d-4a84-849e-0f71d9a37b47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '879bcd89211a432ab36bd5f71b54e8a1'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:46.338961 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-5d34f8d0-288d-4a84-849e-0f71d9a37b47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '879bcd89211a432ab36bd5f71b54e8a1'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:46.340764 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-5d34f8d0-288d-4a84-849e-0f71d9a37b47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '879bcd89211a432ab36bd5f71b54e8a1'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:46.348454 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-979d4491-f85c-4293-a904-a9bd30023b4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1e26c38b9fa443fbfc2d251147a4017'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:46.349321 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-979d4491-f85c-4293-a904-a9bd30023b4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1e26c38b9fa443fbfc2d251147a4017'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:46.358570 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-5d34f8d0-288d-4a84-849e-0f71d9a37b47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '879bcd89211a432ab36bd5f71b54e8a1'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:46.359508 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-5d34f8d0-288d-4a84-849e-0f71d9a37b47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '879bcd89211a432ab36bd5f71b54e8a1'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:46.410449 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:46.418412 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-979d4491-f85c-4293-a904-a9bd30023b4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1e26c38b9fa443fbfc2d251147a4017'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:46.424445 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 840/1704] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:18:45 2022] DELETE /identity/v3/users/879bcd89211a432ab36bd5f71b54e8a1 => generated 0 bytes in 649 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:46.428187 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:46.429829 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288c910-50b0-4d0f-9464-3cca9b0cb07d None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:46.430406 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288c910-50b0-4d0f-9464-3cca9b0cb07d None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:46.430685 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288c910-50b0-4d0f-9464-3cca9b0cb07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:46.460706 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 865/1705] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:18:46 2022] DELETE /identity/v3/users/f1e26c38b9fa443fbfc2d251147a4017 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:46.474505 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-3288c910-50b0-4d0f-9464-3cca9b0cb07d None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:46.476082 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3288c910-50b0-4d0f-9464-3cca9b0cb07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:46.647652 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 841/1706] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:46.669591 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d4491-f85c-4293-a904-a9bd30023b4b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:46.671580 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-979d4491-f85c-4293-a904-a9bd30023b4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:46.838861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d4491-f85c-4293-a904-a9bd30023b4b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:46.861664 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9199fb7d-d3ac-4720-b19d-5311c29e6a05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:46.863215 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9199fb7d-d3ac-4720-b19d-5311c29e6a05 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:46.865119 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9199fb7d-d3ac-4720-b19d-5311c29e6a05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:46.865575 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9199fb7d-d3ac-4720-b19d-5311c29e6a05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:46.893980 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9199fb7d-d3ac-4720-b19d-5311c29e6a05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:46.895987 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9199fb7d-d3ac-4720-b19d-5311c29e6a05 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:46.942516 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 866/1707] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:18:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:18:47.343500 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3288c910-50b0-4d0f-9464-3cca9b0cb07d None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:47.345584 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3288c910-50b0-4d0f-9464-3cca9b0cb07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:47.372290 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3288c910-50b0-4d0f-9464-3cca9b0cb07d None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:47.396037 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db676db-99e7-4cd1-a0bb-8946608efb68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:47.396702 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db676db-99e7-4cd1-a0bb-8946608efb68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:47.396978 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db676db-99e7-4cd1-a0bb-8946608efb68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:47.397239 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db676db-99e7-4cd1-a0bb-8946608efb68 admin admin] PATH_INFO: `/v3/projects/7e3c1e9b224e45f0978ba15ea2308094` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:47.401031 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db676db-99e7-4cd1-a0bb-8946608efb68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e3c1e9b224e45f0978ba15ea2308094)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:47.403351 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db676db-99e7-4cd1-a0bb-8946608efb68 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:47.523381 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0db676db-99e7-4cd1-a0bb-8946608efb68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e3c1e9b224e45f0978ba15ea2308094', 'request_id': 'req-0db676db-99e7-4cd1-a0bb-8946608efb68'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:47.538224 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0db676db-99e7-4cd1-a0bb-8946608efb68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e3c1e9b224e45f0978ba15ea2308094', 'request_id': 'req-0db676db-99e7-4cd1-a0bb-8946608efb68'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:47.549344 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0db676db-99e7-4cd1-a0bb-8946608efb68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e3c1e9b224e45f0978ba15ea2308094', 'request_id': 'req-0db676db-99e7-4cd1-a0bb-8946608efb68'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:47.573567 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0db676db-99e7-4cd1-a0bb-8946608efb68 admin admin] The token cache is being invalidate because project 7e3c1e9b224e45f0978ba15ea2308094 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:18:47.574083 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0db676db-99e7-4cd1-a0bb-8946608efb68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:47.575026 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0db676db-99e7-4cd1-a0bb-8946608efb68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:47.578784 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 842/1708] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:18:47 2022] DELETE /identity/v3/projects/7e3c1e9b224e45f0978ba15ea2308094 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:47.583787 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb11f4-fe63-4840-a5d2-01f50223a446 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:47.584298 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb11f4-fe63-4840-a5d2-01f50223a446 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:47.584702 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb11f4-fe63-4840-a5d2-01f50223a446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:47.606686 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-8bcb11f4-fe63-4840-a5d2-01f50223a446 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:47.608212 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8bcb11f4-fe63-4840-a5d2-01f50223a446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:47.784496 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 867/1709] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:47.795786 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b903b4-1c97-45a7-a826-5d85eae28538 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:47.797851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b903b4-1c97-45a7-a826-5d85eae28538 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:47.798277 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b903b4-1c97-45a7-a826-5d85eae28538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:47.842817 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-29b903b4-1c97-45a7-a826-5d85eae28538 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:47.844289 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-29b903b4-1c97-45a7-a826-5d85eae28538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:47.866657 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb11f4-fe63-4840-a5d2-01f50223a446 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:47.878442 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 843/1710] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:47.881283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb11f4-fe63-4840-a5d2-01f50223a446 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:47.901839 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de843143-fead-4fa3-882a-3d27b50591e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:47.902778 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de843143-fead-4fa3-882a-3d27b50591e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:47.903239 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de843143-fead-4fa3-882a-3d27b50591e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:47.903644 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de843143-fead-4fa3-882a-3d27b50591e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:47.904456 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de843143-fead-4fa3-882a-3d27b50591e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:18:47.904931 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de843143-fead-4fa3-882a-3d27b50591e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:47.907044 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de843143-fead-4fa3-882a-3d27b50591e4 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:47.929171 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 868/1711] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:18:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:18:47.934583 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b903b4-1c97-45a7-a826-5d85eae28538 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:47.946239 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b903b4-1c97-45a7-a826-5d85eae28538 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:47.951129 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec95ac-c0f1-41f2-8562-df8c27338538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:47.951738 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec95ac-c0f1-41f2-8562-df8c27338538 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:47.952014 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec95ac-c0f1-41f2-8562-df8c27338538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:47.952284 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec95ac-c0f1-41f2-8562-df8c27338538 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:47.952933 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ec95ac-c0f1-41f2-8562-df8c27338538 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:47.955780 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ec95ac-c0f1-41f2-8562-df8c27338538 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:48.013939 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 844/1712] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:18:47 2022] POST /identity/v3/projects => generated 381 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:18:48.018906 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de843143-fead-4fa3-882a-3d27b50591e4 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:48.033581 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de843143-fead-4fa3-882a-3d27b50591e4 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:48.142974 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa5bbc-6d09-43ec-b154-e6395020c84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:48.144092 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa5bbc-6d09-43ec-b154-e6395020c84e admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:48.144533 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa5bbc-6d09-43ec-b154-e6395020c84e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:48.144948 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa5bbc-6d09-43ec-b154-e6395020c84e admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:48.148370 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baa5bbc-6d09-43ec-b154-e6395020c84e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:48.150922 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baa5bbc-6d09-43ec-b154-e6395020c84e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:48.199167 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0aaab1-c03f-4bbe-9454-6ab5fb6e1218 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:48.199723 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0aaab1-c03f-4bbe-9454-6ab5fb6e1218 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:48.200735 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0aaab1-c03f-4bbe-9454-6ab5fb6e1218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:48.213677 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 869/1713] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:18:48 2022] POST /identity/v3/users => generated 291 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:18:48.215089 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa5bbc-6d09-43ec-b154-e6395020c84e admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:48.237651 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa5bbc-6d09-43ec-b154-e6395020c84e admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:48.253155 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2534f-8069-457b-b61d-4262304b0b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:48.253780 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2534f-8069-457b-b61d-4262304b0b3b admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:48.254075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2534f-8069-457b-b61d-4262304b0b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:48.254344 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2534f-8069-457b-b61d-4262304b0b3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:48.255004 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c2534f-8069-457b-b61d-4262304b0b3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:48.256618 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c2534f-8069-457b-b61d-4262304b0b3b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:48.277862 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-1f0aaab1-c03f-4bbe-9454-6ab5fb6e1218 None None] MFA Rules not processed for user `af50ca6347404cdf8eb213d27053af16`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:48.280907 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-1f0aaab1-c03f-4bbe-9454-6ab5fb6e1218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:48.293695 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 870/1714] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:18:48 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:18:48.295567 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2534f-8069-457b-b61d-4262304b0b3b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:48.308126 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2534f-8069-457b-b61d-4262304b0b3b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:48.313984 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca342b-8dab-4823-a1be-a09d0cc087b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:48.315365 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca342b-8dab-4823-a1be-a09d0cc087b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:48.315848 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca342b-8dab-4823-a1be-a09d0cc087b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:48.316534 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca342b-8dab-4823-a1be-a09d0cc087b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:48.317733 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca342b-8dab-4823-a1be-a09d0cc087b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:48.320093 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca342b-8dab-4823-a1be-a09d0cc087b9 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:48.322727 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:48.338534 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 871/1715] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:18:48 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:18:48.340751 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca342b-8dab-4823-a1be-a09d0cc087b9 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:48.342542 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4bca342b-8dab-4823-a1be-a09d0cc087b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:48.357191 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca342b-8dab-4823-a1be-a09d0cc087b9 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:48.366939 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e28b07-dde6-4a94-bcfe-e5fa77c12d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:48.368042 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e28b07-dde6-4a94-bcfe-e5fa77c12d78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:48.368400 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e28b07-dde6-4a94-bcfe-e5fa77c12d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:48.368872 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e28b07-dde6-4a94-bcfe-e5fa77c12d78 admin admin] PATH_INFO: `/v3/projects/d2582c3ad1214f6cb9fdffeabd950472` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:48.372344 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e28b07-dde6-4a94-bcfe-e5fa77c12d78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2582c3ad1214f6cb9fdffeabd950472)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:48.375563 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e28b07-dde6-4a94-bcfe-e5fa77c12d78 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:48.455425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-59e28b07-dde6-4a94-bcfe-e5fa77c12d78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2582c3ad1214f6cb9fdffeabd950472', 'request_id': 'req-59e28b07-dde6-4a94-bcfe-e5fa77c12d78'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:48.475809 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:48.488810 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-59e28b07-dde6-4a94-bcfe-e5fa77c12d78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2582c3ad1214f6cb9fdffeabd950472', 'request_id': 'req-59e28b07-dde6-4a94-bcfe-e5fa77c12d78'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:48.507452 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-59e28b07-dde6-4a94-bcfe-e5fa77c12d78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2582c3ad1214f6cb9fdffeabd950472', 'request_id': 'req-59e28b07-dde6-4a94-bcfe-e5fa77c12d78'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:48.539552 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 845/1716] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:48 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:18:48.539552 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0aaab1-c03f-4bbe-9454-6ab5fb6e1218 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:48.541604 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-59e28b07-dde6-4a94-bcfe-e5fa77c12d78 admin admin] The token cache is being invalidate because project d2582c3ad1214f6cb9fdffeabd950472 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:18:48.541902 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-59e28b07-dde6-4a94-bcfe-e5fa77c12d78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:48.548667 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-59e28b07-dde6-4a94-bcfe-e5fa77c12d78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:48.550952 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 872/1717] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:18:48 2022] DELETE /identity/v3/projects/d2582c3ad1214f6cb9fdffeabd950472 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:48.553179 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e28b07-dde6-4a94-bcfe-e5fa77c12d78 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:48.554962 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-59e28b07-dde6-4a94-bcfe-e5fa77c12d78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:48.560479 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0aaab1-c03f-4bbe-9454-6ab5fb6e1218 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:48.680650 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e3c9bb-1637-4ffa-9563-3b868615d805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:48.682352 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3c9bb-1637-4ffa-9563-3b868615d805 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:48.686495 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3c9bb-1637-4ffa-9563-3b868615d805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:48.686836 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3c9bb-1637-4ffa-9563-3b868615d805 admin admin] PATH_INFO: `/v3/projects/ad793b5517824686abbab84a493d792e/users/67b6a742d9e64a0e9894d33bd46dce2b/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:48.722098 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e3c9bb-1637-4ffa-9563-3b868615d805 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad793b5517824686abbab84a493d792e, user_id=67b6a742d9e64a0e9894d33bd46dce2b, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:48.724878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e3c9bb-1637-4ffa-9563-3b868615d805 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:48.753065 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 846/1718] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:18:48 2022] PUT /identity/v3/projects/ad793b5517824686abbab84a493d792e/users/67b6a742d9e64a0e9894d33bd46dce2b/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:48.755435 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e3c9bb-1637-4ffa-9563-3b868615d805 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:48.762726 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c7e3c9bb-1637-4ffa-9563-3b868615d805 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:48.766828 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e28b07-dde6-4a94-bcfe-e5fa77c12d78 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:48.876380 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a74ec9-961f-4636-a941-9a9bdf875d9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:48.877892 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a74ec9-961f-4636-a941-9a9bdf875d9f service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:48.878340 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a74ec9-961f-4636-a941-9a9bdf875d9f service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:48.878741 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a74ec9-961f-4636-a941-9a9bdf875d9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:48.879707 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e3c9bb-1637-4ffa-9563-3b868615d805 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:48.881234 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f8a74ec9-961f-4636-a941-9a9bdf875d9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:48.887966 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd522cde-5b6d-4804-b1f0-bfba3722684b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:48.889273 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd522cde-5b6d-4804-b1f0-bfba3722684b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:48.889575 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd522cde-5b6d-4804-b1f0-bfba3722684b admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:48.889947 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd522cde-5b6d-4804-b1f0-bfba3722684b admin admin] PATH_INFO: `/v3/users/f2be66fc9b3146b6bfecbcc99f60969f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:48.891942 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd522cde-5b6d-4804-b1f0-bfba3722684b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2be66fc9b3146b6bfecbcc99f60969f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:48.894194 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd522cde-5b6d-4804-b1f0-bfba3722684b admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:48.899748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:48.975694 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-bd522cde-5b6d-4804-b1f0-bfba3722684b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2be66fc9b3146b6bfecbcc99f60969f'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:48.987052 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a74ec9-961f-4636-a941-9a9bdf875d9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:48.988014 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-bd522cde-5b6d-4804-b1f0-bfba3722684b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2be66fc9b3146b6bfecbcc99f60969f'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:48.989017 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-bd522cde-5b6d-4804-b1f0-bfba3722684b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2be66fc9b3146b6bfecbcc99f60969f'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:48.990448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a74ec9-961f-4636-a941-9a9bdf875d9f service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:49.003157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 873/1719] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:18:48 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 05:18:49.004797 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d60910-6827-4496-8bcc-62e42ab1b8f6 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:49.005347 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d60910-6827-4496-8bcc-62e42ab1b8f6 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:49.005903 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d60910-6827-4496-8bcc-62e42ab1b8f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:49.041078 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-bd522cde-5b6d-4804-b1f0-bfba3722684b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2be66fc9b3146b6bfecbcc99f60969f'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:49.041690 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-bd522cde-5b6d-4804-b1f0-bfba3722684b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2be66fc9b3146b6bfecbcc99f60969f'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:49.051038 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:49.078390 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 847/1720] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:18:48 2022] DELETE /identity/v3/users/f2be66fc9b3146b6bfecbcc99f60969f => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:49.078635 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-65d60910-6827-4496-8bcc-62e42ab1b8f6 None None] MFA Rules not processed for user `67b6a742d9e64a0e9894d33bd46dce2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:49.079796 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-65d60910-6827-4496-8bcc-62e42ab1b8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:49.237317 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d5865-3846-4fbf-ab71-408bd2c937b4 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:49.239265 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d5865-3846-4fbf-ab71-408bd2c937b4 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:49.239707 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d5865-3846-4fbf-ab71-408bd2c937b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:49.273787 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 874/1721] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:49.286525 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d60910-6827-4496-8bcc-62e42ab1b8f6 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:49.290660 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-65d60910-6827-4496-8bcc-62e42ab1b8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:49.308903 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-039d5865-3846-4fbf-ab71-408bd2c937b4 None None] MFA Rules not processed for user `4b07cff5bc3a4e68b62c4c5d3e798732`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:49.311473 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-039d5865-3846-4fbf-ab71-408bd2c937b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:49.481586 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d60910-6827-4496-8bcc-62e42ab1b8f6 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:49.486977 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 848/1722] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:49.516344 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76816c52-9471-4ec9-8d58-f44f9cab1f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:49.517069 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76816c52-9471-4ec9-8d58-f44f9cab1f1d admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:49.518790 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76816c52-9471-4ec9-8d58-f44f9cab1f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:49.519096 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76816c52-9471-4ec9-8d58-f44f9cab1f1d admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:49.521668 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76816c52-9471-4ec9-8d58-f44f9cab1f1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:49.523808 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76816c52-9471-4ec9-8d58-f44f9cab1f1d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:49.597081 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 875/1723] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:18:49 2022] POST /identity/v3/projects => generated 381 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:18:49.598503 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d5865-3846-4fbf-ab71-408bd2c937b4 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:49.624308 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d5865-3846-4fbf-ab71-408bd2c937b4 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:49.712095 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab8c749-77f8-4a04-bb33-1c26785ca43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:49.720586 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab8c749-77f8-4a04-bb33-1c26785ca43d admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:49.720977 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab8c749-77f8-4a04-bb33-1c26785ca43d admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:49.721184 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab8c749-77f8-4a04-bb33-1c26785ca43d admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:49.722416 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab8c749-77f8-4a04-bb33-1c26785ca43d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:49.725225 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab8c749-77f8-4a04-bb33-1c26785ca43d admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:49.774517 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 849/1724] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:18:49 2022] POST /identity/v3/users => generated 297 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:18:49.777454 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76816c52-9471-4ec9-8d58-f44f9cab1f1d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:49.794078 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab8c749-77f8-4a04-bb33-1c26785ca43d admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:49.795892 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-eab8c749-77f8-4a04-bb33-1c26785ca43d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:49.803655 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76816c52-9471-4ec9-8d58-f44f9cab1f1d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:49.813105 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7872351c-015f-4846-8959-e2a4b9d4f2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:49.813652 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872351c-015f-4846-8959-e2a4b9d4f2fe admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:49.813931 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872351c-015f-4846-8959-e2a4b9d4f2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:49.817406 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872351c-015f-4846-8959-e2a4b9d4f2fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:49.818065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7872351c-015f-4846-8959-e2a4b9d4f2fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:49.819402 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab8c749-77f8-4a04-bb33-1c26785ca43d admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:49.824331 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7872351c-015f-4846-8959-e2a4b9d4f2fe admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:49.835103 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d512cc9-8b5c-4b34-b1a5-05e66b3e0ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:49.835768 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d512cc9-8b5c-4b34-b1a5-05e66b3e0ada admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:49.836050 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d512cc9-8b5c-4b34-b1a5-05e66b3e0ada admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:49.836320 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d512cc9-8b5c-4b34-b1a5-05e66b3e0ada admin admin] PATH_INFO: `/v3/projects/2600bf39f9e04b11a58106167c541350` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:49.842906 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d512cc9-8b5c-4b34-b1a5-05e66b3e0ada admin admin] RBAC: Authorizing `identity:delete_project(project_id=2600bf39f9e04b11a58106167c541350)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:49.845153 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d512cc9-8b5c-4b34-b1a5-05e66b3e0ada admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:49.872240 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 876/1725] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:18:49.873711 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7872351c-015f-4846-8959-e2a4b9d4f2fe admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:49.888756 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7872351c-015f-4846-8959-e2a4b9d4f2fe admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:49.905746 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab06af0-1205-41bf-b3c3-1a16908309ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:49.906607 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab06af0-1205-41bf-b3c3-1a16908309ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:49.906995 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab06af0-1205-41bf-b3c3-1a16908309ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:49.907372 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab06af0-1205-41bf-b3c3-1a16908309ff admin admin] PATH_INFO: `/v3/projects/2fd6349fbb4448eabd9d179580d41718/users/6c909fdb62f04edfb9e4953b24ee815f/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:49.944139 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab06af0-1205-41bf-b3c3-1a16908309ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fd6349fbb4448eabd9d179580d41718, user_id=6c909fdb62f04edfb9e4953b24ee815f, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:49.948387 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab06af0-1205-41bf-b3c3-1a16908309ff admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:49.952644 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:49.966657 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-2d512cc9-8b5c-4b34-b1a5-05e66b3e0ada admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2600bf39f9e04b11a58106167c541350', 'request_id': 'req-2d512cc9-8b5c-4b34-b1a5-05e66b3e0ada'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:49.978434 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 877/1726] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:18:49 2022] PUT /identity/v3/projects/2fd6349fbb4448eabd9d179580d41718/users/6c909fdb62f04edfb9e4953b24ee815f/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:49.979824 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab06af0-1205-41bf-b3c3-1a16908309ff admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:49.988828 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-2d512cc9-8b5c-4b34-b1a5-05e66b3e0ada admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2600bf39f9e04b11a58106167c541350', 'request_id': 'req-2d512cc9-8b5c-4b34-b1a5-05e66b3e0ada'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:50.007185 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-2d512cc9-8b5c-4b34-b1a5-05e66b3e0ada admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2600bf39f9e04b11a58106167c541350', 'request_id': 'req-2d512cc9-8b5c-4b34-b1a5-05e66b3e0ada'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:50.041962 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-2d512cc9-8b5c-4b34-b1a5-05e66b3e0ada admin admin] The token cache is being invalidate because project 2600bf39f9e04b11a58106167c541350 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:18:50.042365 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-2d512cc9-8b5c-4b34-b1a5-05e66b3e0ada admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:50.043226 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-2d512cc9-8b5c-4b34-b1a5-05e66b3e0ada admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:50.046939 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 850/1727] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:18:49 2022] DELETE /identity/v3/projects/2600bf39f9e04b11a58106167c541350 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:50.052211 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0979743-a01b-4b1a-a6c5-c5e0d5b4089c None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:50.052495 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0979743-a01b-4b1a-a6c5-c5e0d5b4089c None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:50.052805 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0979743-a01b-4b1a-a6c5-c5e0d5b4089c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:50.079936 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=88,keystone:DELETE=8,keystone:INSERT=4 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:50.092323 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab06af0-1205-41bf-b3c3-1a16908309ff admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:50.097650 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d17cb52-885e-4758-9576-1c0e97a29a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:50.098437 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17cb52-885e-4758-9576-1c0e97a29a5c admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:50.098822 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17cb52-885e-4758-9576-1c0e97a29a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:50.099212 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17cb52-885e-4758-9576-1c0e97a29a5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:50.099972 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d17cb52-885e-4758-9576-1c0e97a29a5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:50.101677 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d17cb52-885e-4758-9576-1c0e97a29a5c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:50.111618 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-d0979743-a01b-4b1a-a6c5-c5e0d5b4089c None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:50.112989 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-d0979743-a01b-4b1a-a6c5-c5e0d5b4089c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:50.124344 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 878/1728] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:18:50.126108 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d17cb52-885e-4758-9576-1c0e97a29a5c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:50.127928 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1d17cb52-885e-4758-9576-1c0e97a29a5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:50.144520 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 851/1729] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:18:50.144730 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d17cb52-885e-4758-9576-1c0e97a29a5c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:50.147116 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bedab85-5c8e-406d-8802-ecec37a95c16 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:50.147438 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bedab85-5c8e-406d-8802-ecec37a95c16 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:50.147731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bedab85-5c8e-406d-8802-ecec37a95c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:50.150519 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d171d1f-3f31-42c4-8493-ab29e9abcaee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:50.151740 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d171d1f-3f31-42c4-8493-ab29e9abcaee admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:50.152191 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d171d1f-3f31-42c4-8493-ab29e9abcaee admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:50.152635 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d171d1f-3f31-42c4-8493-ab29e9abcaee admin admin] PATH_INFO: `/v3/projects/2fd6349fbb4448eabd9d179580d41718/users/6c909fdb62f04edfb9e4953b24ee815f/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:50.156356 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d171d1f-3f31-42c4-8493-ab29e9abcaee admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fd6349fbb4448eabd9d179580d41718, user_id=6c909fdb62f04edfb9e4953b24ee815f, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:50.162921 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d171d1f-3f31-42c4-8493-ab29e9abcaee admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:50.176612 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 879/1730] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:18:50 2022] PUT /identity/v3/projects/2fd6349fbb4448eabd9d179580d41718/users/6c909fdb62f04edfb9e4953b24ee815f/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:50.188012 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc73c9bf-ee68-429f-86c4-64fb10b98331 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:50.188301 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc73c9bf-ee68-429f-86c4-64fb10b98331 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:50.188570 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc73c9bf-ee68-429f-86c4-64fb10b98331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:50.204768 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-5bedab85-5c8e-406d-8802-ecec37a95c16 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:50.206023 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-5bedab85-5c8e-406d-8802-ecec37a95c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:50.286686 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-fc73c9bf-ee68-429f-86c4-64fb10b98331 None None] MFA Rules not processed for user `6c909fdb62f04edfb9e4953b24ee815f`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:50.287918 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fc73c9bf-ee68-429f-86c4-64fb10b98331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:50.345501 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 852/1731] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:18:50.346798 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bedab85-5c8e-406d-8802-ecec37a95c16 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:50.362227 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bedab85-5c8e-406d-8802-ecec37a95c16 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:50.369885 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e10e2-be4b-4d37-8e7c-bb758e0f545a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:50.370782 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e10e2-be4b-4d37-8e7c-bb758e0f545a admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:50.371321 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e10e2-be4b-4d37-8e7c-bb758e0f545a admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:50.372337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e10e2-be4b-4d37-8e7c-bb758e0f545a admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:50.373801 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e10e2-be4b-4d37-8e7c-bb758e0f545a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:18:50.374365 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e10e2-be4b-4d37-8e7c-bb758e0f545a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:50.377044 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e10e2-be4b-4d37-8e7c-bb758e0f545a admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:50.395495 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 853/1732] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:18:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:18:50.399533 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e10e2-be4b-4d37-8e7c-bb758e0f545a admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:50.416050 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e10e2-be4b-4d37-8e7c-bb758e0f545a admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:50.431694 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2347d840-5a3f-4289-9765-9b09d188b6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:50.433727 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347d840-5a3f-4289-9765-9b09d188b6b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:50.434027 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347d840-5a3f-4289-9765-9b09d188b6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:50.434933 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347d840-5a3f-4289-9765-9b09d188b6b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:50.437523 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2347d840-5a3f-4289-9765-9b09d188b6b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:50.442208 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2347d840-5a3f-4289-9765-9b09d188b6b9 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:50.495490 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 854/1733] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:18:50 2022] POST /identity/v3/projects => generated 389 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:18:50.500453 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2347d840-5a3f-4289-9765-9b09d188b6b9 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:50.502264 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 880/1734] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:50 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:50.503326 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc73c9bf-ee68-429f-86c4-64fb10b98331 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:50.511646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fc73c9bf-ee68-429f-86c4-64fb10b98331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:50.604353 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2347d840-5a3f-4289-9765-9b09d188b6b9 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:50.615100 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74078a03-7015-4bb7-b55d-8d900538de8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:50.616087 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74078a03-7015-4bb7-b55d-8d900538de8c admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:50.616578 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74078a03-7015-4bb7-b55d-8d900538de8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:50.617066 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74078a03-7015-4bb7-b55d-8d900538de8c admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:50.618431 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74078a03-7015-4bb7-b55d-8d900538de8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:50.621113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74078a03-7015-4bb7-b55d-8d900538de8c admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:50.626246 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc73c9bf-ee68-429f-86c4-64fb10b98331 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:50.647151 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:50.676639 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 855/1735] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:18:50 2022] POST /identity/v3/users => generated 295 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:18:50.679992 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74078a03-7015-4bb7-b55d-8d900538de8c admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:50.713695 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74078a03-7015-4bb7-b55d-8d900538de8c admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:50.725798 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c2a3f2-9d0c-4ca3-b79a-50c7ed35acdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:50.726946 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2a3f2-9d0c-4ca3-b79a-50c7ed35acdd admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:50.727485 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2a3f2-9d0c-4ca3-b79a-50c7ed35acdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:50.728404 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2a3f2-9d0c-4ca3-b79a-50c7ed35acdd admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:50.729071 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c2a3f2-9d0c-4ca3-b79a-50c7ed35acdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:50.731836 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c2a3f2-9d0c-4ca3-b79a-50c7ed35acdd admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:50.743183 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13587ea3-6b8e-4231-91de-dd25e43c0344 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:50.743815 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13587ea3-6b8e-4231-91de-dd25e43c0344 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:50.744091 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13587ea3-6b8e-4231-91de-dd25e43c0344 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:50.744355 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13587ea3-6b8e-4231-91de-dd25e43c0344 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:50.751621 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 856/1736] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:18:50 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:18:50.753468 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c2a3f2-9d0c-4ca3-b79a-50c7ed35acdd admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:50.779946 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c2a3f2-9d0c-4ca3-b79a-50c7ed35acdd admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:50.791562 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9573593f-0a49-4014-b80d-d0a957347c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:50.794478 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9573593f-0a49-4014-b80d-d0a957347c9e admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:50.798115 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9573593f-0a49-4014-b80d-d0a957347c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:50.798485 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9573593f-0a49-4014-b80d-d0a957347c9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:50.800340 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9573593f-0a49-4014-b80d-d0a957347c9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:50.805095 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9573593f-0a49-4014-b80d-d0a957347c9e admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:50.831482 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 857/1737] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:18:50 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:18:50.833273 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9573593f-0a49-4014-b80d-d0a957347c9e admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:50.847599 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9573593f-0a49-4014-b80d-d0a957347c9e admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:50.851424 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13587ea3-6b8e-4231-91de-dd25e43c0344 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:50.852999 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13587ea3-6b8e-4231-91de-dd25e43c0344 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:50.855618 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f038cc9e-b275-466f-a35d-36d7e783f481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:50.856799 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038cc9e-b275-466f-a35d-36d7e783f481 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:50.857272 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038cc9e-b275-466f-a35d-36d7e783f481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:50.857746 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038cc9e-b275-466f-a35d-36d7e783f481 admin admin] PATH_INFO: `/v3/projects/30107529c3c7474f90e50f5368d6620a/users/e164cf530d3946429bc3d648a6ca45c1/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:50.869634 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 881/1738] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:18:50 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:18:50.878256 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f038cc9e-b275-466f-a35d-36d7e783f481 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30107529c3c7474f90e50f5368d6620a, user_id=e164cf530d3946429bc3d648a6ca45c1, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:50.881318 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f038cc9e-b275-466f-a35d-36d7e783f481 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:50.917176 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 858/1739] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:18:50 2022] PUT /identity/v3/projects/30107529c3c7474f90e50f5368d6620a/users/e164cf530d3946429bc3d648a6ca45c1/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:50.924164 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e1161-ae08-4975-9d03-268dbdba6b58 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:50.924679 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e1161-ae08-4975-9d03-268dbdba6b58 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:50.925049 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e1161-ae08-4975-9d03-268dbdba6b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:50.957528 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b20e1161-ae08-4975-9d03-268dbdba6b58 None None] MFA Rules not processed for user `e164cf530d3946429bc3d648a6ca45c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:50.958909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b20e1161-ae08-4975-9d03-268dbdba6b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:51.026624 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f038cc9e-b275-466f-a35d-36d7e783f481 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:51.043940 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:51.127008 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 882/1740] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:50 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:18:51.131437 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20e1161-ae08-4975-9d03-268dbdba6b58 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:51.201022 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f038cc9e-b275-466f-a35d-36d7e783f481 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:51.208374 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522c333-3042-4ecf-b933-a803d0d6c41a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:51.209179 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522c333-3042-4ecf-b933-a803d0d6c41a service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:51.209450 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522c333-3042-4ecf-b933-a803d0d6c41a service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:51.209748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522c333-3042-4ecf-b933-a803d0d6c41a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:51.219541 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c522c333-3042-4ecf-b933-a803d0d6c41a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:51.271498 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:51.273396 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20e1161-ae08-4975-9d03-268dbdba6b58 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:51.286109 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7df0e6b-0385-4873-b338-d9ed1c8ba3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:51.286905 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df0e6b-0385-4873-b338-d9ed1c8ba3d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:51.287358 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df0e6b-0385-4873-b338-d9ed1c8ba3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:51.287797 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df0e6b-0385-4873-b338-d9ed1c8ba3d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:51.288539 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7df0e6b-0385-4873-b338-d9ed1c8ba3d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:51.296028 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7df0e6b-0385-4873-b338-d9ed1c8ba3d9 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:51.363483 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c522c333-3042-4ecf-b933-a803d0d6c41a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:51.365341 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c522c333-3042-4ecf-b933-a803d0d6c41a service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:51.377810 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 883/1741] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:18:51 2022] POST /identity/v3/projects => generated 389 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:18:51.381975 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7df0e6b-0385-4873-b338-d9ed1c8ba3d9 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:51.386607 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 859/1742] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:18:51 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:18:51.471757 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7df0e6b-0385-4873-b338-d9ed1c8ba3d9 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:51.507041 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d515a589-433b-4386-aab0-e70273bc7668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:51.509717 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d515a589-433b-4386-aab0-e70273bc7668 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:51.511421 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d515a589-433b-4386-aab0-e70273bc7668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:51.512149 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d515a589-433b-4386-aab0-e70273bc7668 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:51.514166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d515a589-433b-4386-aab0-e70273bc7668 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:51.517784 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d515a589-433b-4386-aab0-e70273bc7668 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:51.572892 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 884/1743] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:18:51 2022] POST /identity/v3/users => generated 301 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:18:51.576414 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522c333-3042-4ecf-b933-a803d0d6c41a service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:51.598207 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522c333-3042-4ecf-b933-a803d0d6c41a service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:51.607735 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a77a9c-f75b-4a6f-bbe1-ac73db44ec07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:51.609093 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a77a9c-f75b-4a6f-bbe1-ac73db44ec07 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:51.609725 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a77a9c-f75b-4a6f-bbe1-ac73db44ec07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:51.610262 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a77a9c-f75b-4a6f-bbe1-ac73db44ec07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:51.612460 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a77a9c-f75b-4a6f-bbe1-ac73db44ec07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:51.636701 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a77a9c-f75b-4a6f-bbe1-ac73db44ec07 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:51.667545 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 860/1744] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:18:51 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:18:51.673189 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d515a589-433b-4386-aab0-e70273bc7668 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:51.696515 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d515a589-433b-4386-aab0-e70273bc7668 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:51.713371 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1b296-c6f9-40c1-af60-657179894619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:51.714416 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1b296-c6f9-40c1-af60-657179894619 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:51.714697 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1b296-c6f9-40c1-af60-657179894619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:51.714968 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1b296-c6f9-40c1-af60-657179894619 admin admin] PATH_INFO: `/v3/projects/00e7a2da64bd4c29a237e87bff03c2e6/users/60842f8a8ae24b12bd975cea0abb9339/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:51.744576 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a1b296-c6f9-40c1-af60-657179894619 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00e7a2da64bd4c29a237e87bff03c2e6, user_id=60842f8a8ae24b12bd975cea0abb9339, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:51.747138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a1b296-c6f9-40c1-af60-657179894619 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:51.767178 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 885/1745] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:18:51 2022] PUT /identity/v3/projects/00e7a2da64bd4c29a237e87bff03c2e6/users/60842f8a8ae24b12bd975cea0abb9339/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:51.771733 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a77a9c-f75b-4a6f-bbe1-ac73db44ec07 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:51.795484 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a77a9c-f75b-4a6f-bbe1-ac73db44ec07 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:51.875377 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c18ff-6cd2-4b4c-8470-33614cc0c0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:51.876532 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c18ff-6cd2-4b4c-8470-33614cc0c0a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:51.877412 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c18ff-6cd2-4b4c-8470-33614cc0c0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:51.895660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c18ff-6cd2-4b4c-8470-33614cc0c0a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:51.896735 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54c18ff-6cd2-4b4c-8470-33614cc0c0a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:51.902310 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54c18ff-6cd2-4b4c-8470-33614cc0c0a2 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:51.920609 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 861/1746] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:18:51 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:18:51.921618 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1b296-c6f9-40c1-af60-657179894619 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:51.951381 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1b296-c6f9-40c1-af60-657179894619 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:51.963363 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83bc315-87ed-48b0-8cb7-5c0d1f866f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:51.964676 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bc315-87ed-48b0-8cb7-5c0d1f866f24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:51.965326 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bc315-87ed-48b0-8cb7-5c0d1f866f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:51.965955 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bc315-87ed-48b0-8cb7-5c0d1f866f24 admin admin] PATH_INFO: `/v3/projects/00e7a2da64bd4c29a237e87bff03c2e6/users/60842f8a8ae24b12bd975cea0abb9339/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:51.969860 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83bc315-87ed-48b0-8cb7-5c0d1f866f24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00e7a2da64bd4c29a237e87bff03c2e6, user_id=60842f8a8ae24b12bd975cea0abb9339, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:51.973636 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83bc315-87ed-48b0-8cb7-5c0d1f866f24 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:51.988133 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 886/1747] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:18:51 2022] PUT /identity/v3/projects/00e7a2da64bd4c29a237e87bff03c2e6/users/60842f8a8ae24b12bd975cea0abb9339/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:18:51.999487 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ed67c-9cac-4a2a-9df1-9654afa7a9e7 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:51.999801 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ed67c-9cac-4a2a-9df1-9654afa7a9e7 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:52.000127 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ed67c-9cac-4a2a-9df1-9654afa7a9e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:52.041890 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-072ed67c-9cac-4a2a-9df1-9654afa7a9e7 None None] MFA Rules not processed for user `60842f8a8ae24b12bd975cea0abb9339`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:52.043356 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-072ed67c-9cac-4a2a-9df1-9654afa7a9e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:52.060922 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7369673-7a44-4360-8570-6a04d76a8480 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:52.061237 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7369673-7a44-4360-8570-6a04d76a8480 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:52.061631 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7369673-7a44-4360-8570-6a04d76a8480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:52.088119 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-c7369673-7a44-4360-8570-6a04d76a8480 None None] MFA Rules not processed for user `af50ca6347404cdf8eb213d27053af16`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:52.089311 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c7369673-7a44-4360-8570-6a04d76a8480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:52.096970 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:52.145388 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 862/1748] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:51 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:52.158966 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ed67c-9cac-4a2a-9df1-9654afa7a9e7 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:52.160524 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-072ed67c-9cac-4a2a-9df1-9654afa7a9e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:52.174046 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 887/1749] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:52 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:52.245640 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ed67c-9cac-4a2a-9df1-9654afa7a9e7 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:52.250708 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ce71f-3a3b-4fac-940e-f46cbbd337e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:52.250882 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ce71f-3a3b-4fac-940e-f46cbbd337e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:52.251337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ce71f-3a3b-4fac-940e-f46cbbd337e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:52.251694 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ce71f-3a3b-4fac-940e-f46cbbd337e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:52.278066 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83ce71f-3a3b-4fac-940e-f46cbbd337e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:52.283701 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83ce71f-3a3b-4fac-940e-f46cbbd337e0 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:52.299914 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 863/1750] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:18:52 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 05:18:52.803397 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7369673-7a44-4360-8570-6a04d76a8480 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:52.940719 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:53.032598 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7369673-7a44-4360-8570-6a04d76a8480 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:53.053313 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c251d7-b0b1-481c-a115-597d0b3f668b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:53.055729 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c251d7-b0b1-481c-a115-597d0b3f668b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:53.057314 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c251d7-b0b1-481c-a115-597d0b3f668b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:53.058314 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c251d7-b0b1-481c-a115-597d0b3f668b admin admin] PATH_INFO: `/v3/users/e164cf530d3946429bc3d648a6ca45c1` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:53.061693 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c251d7-b0b1-481c-a115-597d0b3f668b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e164cf530d3946429bc3d648a6ca45c1)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:53.068816 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c251d7-b0b1-481c-a115-597d0b3f668b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:53.115113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e440193-af9c-44f2-a885-43e560f6ca3c None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:53.115569 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e440193-af9c-44f2-a885-43e560f6ca3c None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:53.116460 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e440193-af9c-44f2-a885-43e560f6ca3c None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:53.117160 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 864/1751] 10.222.0.36 () {54 vars in 912 bytes} [Wed Aug 31 05:18:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 05:18:53.137237 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d806145-3913-4d1e-8cbe-6e6178436cd9 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:53.137467 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d806145-3913-4d1e-8cbe-6e6178436cd9 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:53.137749 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d806145-3913-4d1e-8cbe-6e6178436cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:53.182197 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-12c251d7-b0b1-481c-a115-597d0b3f668b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e164cf530d3946429bc3d648a6ca45c1'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:53.187403 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-7d806145-3913-4d1e-8cbe-6e6178436cd9 None None] MFA Rules not processed for user `090e1409506542f6831e1f65d44aae74`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:53.188913 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-7d806145-3913-4d1e-8cbe-6e6178436cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:53.194955 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-12c251d7-b0b1-481c-a115-597d0b3f668b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e164cf530d3946429bc3d648a6ca45c1'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:53.211844 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-12c251d7-b0b1-481c-a115-597d0b3f668b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e164cf530d3946429bc3d648a6ca45c1'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:53.212825 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-12c251d7-b0b1-481c-a115-597d0b3f668b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e164cf530d3946429bc3d648a6ca45c1'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:53.249470 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-12c251d7-b0b1-481c-a115-597d0b3f668b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e164cf530d3946429bc3d648a6ca45c1'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:53.264146 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 888/1752] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:18:52 2022] DELETE /identity/v3/users/e164cf530d3946429bc3d648a6ca45c1 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:53.265457 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c4a665-7ed6-4c45-b1eb-17dd7d287b8a None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:53.265778 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c4a665-7ed6-4c45-b1eb-17dd7d287b8a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:53.266072 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c4a665-7ed6-4c45-b1eb-17dd7d287b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:53.297733 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 865/1753] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Aug 31 05:18:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 31 05:18:53.299429 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a4969-d554-49f1-bb2d-2f4b4df4c872 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:53.299724 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a4969-d554-49f1-bb2d-2f4b4df4c872 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:53.315695 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a4969-d554-49f1-bb2d-2f4b4df4c872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:53.364216 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-d71a4969-d554-49f1-bb2d-2f4b4df4c872 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:53.372902 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-d71a4969-d554-49f1-bb2d-2f4b4df4c872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:53.392296 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-79c4a665-7ed6-4c45-b1eb-17dd7d287b8a None None] MFA Rules not processed for user `09bf6db88ce8494a937211a814c9ff84`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:53.395819 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-79c4a665-7ed6-4c45-b1eb-17dd7d287b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:53.549218 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 866/1754] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:53.549218 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71a4969-d554-49f1-bb2d-2f4b4df4c872 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:53.562607 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-d71a4969-d554-49f1-bb2d-2f4b4df4c872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:53.588185 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 889/1755] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Aug 31 05:18:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:53.588826 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c4a665-7ed6-4c45-b1eb-17dd7d287b8a None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:53.591604 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-79c4a665-7ed6-4c45-b1eb-17dd7d287b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:53.677195 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:53.748667 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71a4969-d554-49f1-bb2d-2f4b4df4c872 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:53.758854 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:53.762403 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0078a29-84fa-44ab-87f4-100e5529bf90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:53.763312 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0078a29-84fa-44ab-87f4-100e5529bf90 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:53.763664 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0078a29-84fa-44ab-87f4-100e5529bf90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:53.764006 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0078a29-84fa-44ab-87f4-100e5529bf90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:53.792322 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0078a29-84fa-44ab-87f4-100e5529bf90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:53.798094 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0078a29-84fa-44ab-87f4-100e5529bf90 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:53.836063 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 867/1756] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:18:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:18:53.879193 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c4a665-7ed6-4c45-b1eb-17dd7d287b8a None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:53.906536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1569a7d-a9ae-4ea1-89cc-059b46509800 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:53.908708 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1569a7d-a9ae-4ea1-89cc-059b46509800 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:53.911074 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1569a7d-a9ae-4ea1-89cc-059b46509800 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:53.911600 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1569a7d-a9ae-4ea1-89cc-059b46509800 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:53.938485 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1569a7d-a9ae-4ea1-89cc-059b46509800 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:53.942168 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1569a7d-a9ae-4ea1-89cc-059b46509800 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:53.957618 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0078a29-84fa-44ab-87f4-100e5529bf90 service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:53.959522 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-e0078a29-84fa-44ab-87f4-100e5529bf90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:53.980037 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 890/1757] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:18:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:18:54.169455 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0078a29-84fa-44ab-87f4-100e5529bf90 service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:54.188954 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db5b75-3299-4bb2-8fb2-a05037789ff6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9b82080a73d4c4a8ec08b0e67d6d11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:54.189792 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db5b75-3299-4bb2-8fb2-a05037789ff6 service swift] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:54.190314 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db5b75-3299-4bb2-8fb2-a05037789ff6 service swift] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:54.190699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db5b75-3299-4bb2-8fb2-a05037789ff6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:54.230649 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6db5b75-3299-4bb2-8fb2-a05037789ff6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:54.232500 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6db5b75-3299-4bb2-8fb2-a05037789ff6 service swift] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:54.246994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 868/1758] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Aug 31 05:18:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 31 05:18:55.679749 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1569a7d-a9ae-4ea1-89cc-059b46509800 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:55.682061 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c1569a7d-a9ae-4ea1-89cc-059b46509800 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:55.712866 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1569a7d-a9ae-4ea1-89cc-059b46509800 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:55.730218 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e00950e-4604-4268-a9f3-958ed99a1550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:55.733289 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00950e-4604-4268-a9f3-958ed99a1550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:55.734830 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00950e-4604-4268-a9f3-958ed99a1550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:55.736661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00950e-4604-4268-a9f3-958ed99a1550 admin admin] PATH_INFO: `/v3/users/9a56feccbe844aff84e2b35a179a9e16` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:55.744495 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e00950e-4604-4268-a9f3-958ed99a1550 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a56feccbe844aff84e2b35a179a9e16)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:55.754835 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e00950e-4604-4268-a9f3-958ed99a1550 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:55.830455 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee3d8ae-5db8-4c95-871b-966b4ba6e6e3 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:55.830455 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee3d8ae-5db8-4c95-871b-966b4ba6e6e3 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:55.830455 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee3d8ae-5db8-4c95-871b-966b4ba6e6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:55.856181 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-fee3d8ae-5db8-4c95-871b-966b4ba6e6e3 None None] MFA Rules not processed for user `090e1409506542f6831e1f65d44aae74`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:55.857783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-fee3d8ae-5db8-4c95-871b-966b4ba6e6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:55.887955 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 869/1759] 10.222.0.43 () {58 vars in 1015 bytes} [Wed Aug 31 05:18:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:55.912383 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee3d8ae-5db8-4c95-871b-966b4ba6e6e3 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:55.914452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-fee3d8ae-5db8-4c95-871b-966b4ba6e6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:55.928219 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1e00950e-4604-4268-a9f3-958ed99a1550 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a56feccbe844aff84e2b35a179a9e16'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:55.933273 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee3d8ae-5db8-4c95-871b-966b4ba6e6e3 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:55.945843 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1e00950e-4604-4268-a9f3-958ed99a1550 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a56feccbe844aff84e2b35a179a9e16'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:55.948994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d9c2b-fac3-4820-b2e8-15c5399476f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:55.949972 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d9c2b-fac3-4820-b2e8-15c5399476f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:55.950604 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d9c2b-fac3-4820-b2e8-15c5399476f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:55.951232 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d9c2b-fac3-4820-b2e8-15c5399476f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:55.976948 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845d9c2b-fac3-4820-b2e8-15c5399476f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:55.979887 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845d9c2b-fac3-4820-b2e8-15c5399476f3 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:55.982948 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1e00950e-4604-4268-a9f3-958ed99a1550 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a56feccbe844aff84e2b35a179a9e16'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:56.004134 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 870/1760] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:18:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 05:18:56.058658 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1e00950e-4604-4268-a9f3-958ed99a1550 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a56feccbe844aff84e2b35a179a9e16'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:56.127607 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1e00950e-4604-4268-a9f3-958ed99a1550 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a56feccbe844aff84e2b35a179a9e16'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:56.150057 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 891/1761] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:18:55 2022] DELETE /identity/v3/users/9a56feccbe844aff84e2b35a179a9e16 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:56.152075 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b312e84f-a438-492e-af7a-24759ab90e1b None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:56.152693 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b312e84f-a438-492e-af7a-24759ab90e1b None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:56.153281 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b312e84f-a438-492e-af7a-24759ab90e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:56.196006 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-b312e84f-a438-492e-af7a-24759ab90e1b None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:56.199670 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b312e84f-a438-492e-af7a-24759ab90e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:56.333663 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 871/1762] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:56.416954 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e00950e-4604-4268-a9f3-958ed99a1550 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:56.418911 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1e00950e-4604-4268-a9f3-958ed99a1550 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:56.534981 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b312e84f-a438-492e-af7a-24759ab90e1b None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:56.541771 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b312e84f-a438-492e-af7a-24759ab90e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:56.561194 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b312e84f-a438-492e-af7a-24759ab90e1b None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:56.582636 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b26bb3-5c7e-4c86-a2cd-acd2f8482c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:56.583586 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b26bb3-5c7e-4c86-a2cd-acd2f8482c49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:56.584147 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b26bb3-5c7e-4c86-a2cd-acd2f8482c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:56.585184 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b26bb3-5c7e-4c86-a2cd-acd2f8482c49 admin admin] PATH_INFO: `/v3/projects/30107529c3c7474f90e50f5368d6620a` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:56.586962 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b26bb3-5c7e-4c86-a2cd-acd2f8482c49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30107529c3c7474f90e50f5368d6620a)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:56.592010 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b26bb3-5c7e-4c86-a2cd-acd2f8482c49 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:56.613726 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:56.690794 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-63b26bb3-5c7e-4c86-a2cd-acd2f8482c49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30107529c3c7474f90e50f5368d6620a', 'request_id': 'req-63b26bb3-5c7e-4c86-a2cd-acd2f8482c49'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:56.706423 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-63b26bb3-5c7e-4c86-a2cd-acd2f8482c49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30107529c3c7474f90e50f5368d6620a', 'request_id': 'req-63b26bb3-5c7e-4c86-a2cd-acd2f8482c49'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:56.723051 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-63b26bb3-5c7e-4c86-a2cd-acd2f8482c49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30107529c3c7474f90e50f5368d6620a', 'request_id': 'req-63b26bb3-5c7e-4c86-a2cd-acd2f8482c49'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:56.723689 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e00950e-4604-4268-a9f3-958ed99a1550 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:56.769885 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-63b26bb3-5c7e-4c86-a2cd-acd2f8482c49 admin admin] The token cache is being invalidate because project 30107529c3c7474f90e50f5368d6620a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:18:56.770262 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-63b26bb3-5c7e-4c86-a2cd-acd2f8482c49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:56.771205 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-63b26bb3-5c7e-4c86-a2cd-acd2f8482c49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:56.777557 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 872/1763] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:18:56 2022] DELETE /identity/v3/projects/30107529c3c7474f90e50f5368d6620a => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:56.780799 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b26bb3-5c7e-4c86-a2cd-acd2f8482c49 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:56.789968 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-63b26bb3-5c7e-4c86-a2cd-acd2f8482c49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:56.919046 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd3baf-f66f-4e47-a629-4298cc49b54a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:56.920647 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd3baf-f66f-4e47-a629-4298cc49b54a service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:56.920647 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd3baf-f66f-4e47-a629-4298cc49b54a service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:56.920647 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd3baf-f66f-4e47-a629-4298cc49b54a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:56.929403 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c3bd3baf-f66f-4e47-a629-4298cc49b54a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:56.966572 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:57.037690 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b26bb3-5c7e-4c86-a2cd-acd2f8482c49 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:57.060018 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ffd8ee-8c48-4feb-9ab0-37da4cb64cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:57.060784 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffd8ee-8c48-4feb-9ab0-37da4cb64cb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:57.061118 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffd8ee-8c48-4feb-9ab0-37da4cb64cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:57.061397 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffd8ee-8c48-4feb-9ab0-37da4cb64cb5 admin admin] PATH_INFO: `/v3/users/60842f8a8ae24b12bd975cea0abb9339` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:57.063717 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ffd8ee-8c48-4feb-9ab0-37da4cb64cb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60842f8a8ae24b12bd975cea0abb9339)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:57.071395 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ffd8ee-8c48-4feb-9ab0-37da4cb64cb5 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:57.132470 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bd3baf-f66f-4e47-a629-4298cc49b54a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:57.133944 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bd3baf-f66f-4e47-a629-4298cc49b54a service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:57.166356 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 892/1764] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:18:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 750 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:18:57.208952 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f8ffd8ee-8c48-4feb-9ab0-37da4cb64cb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60842f8a8ae24b12bd975cea0abb9339'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:57.221984 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f8ffd8ee-8c48-4feb-9ab0-37da4cb64cb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60842f8a8ae24b12bd975cea0abb9339'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:57.222788 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f8ffd8ee-8c48-4feb-9ab0-37da4cb64cb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60842f8a8ae24b12bd975cea0abb9339'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:57.240564 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f8ffd8ee-8c48-4feb-9ab0-37da4cb64cb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60842f8a8ae24b12bd975cea0abb9339'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:57.241787 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f8ffd8ee-8c48-4feb-9ab0-37da4cb64cb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60842f8a8ae24b12bd975cea0abb9339'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:57.267320 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 873/1765] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:18:56 2022] DELETE /identity/v3/users/60842f8a8ae24b12bd975cea0abb9339 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:57.823650 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8529f87b-53f1-4177-afb3-34cd9e4014ba None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:57.824216 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8529f87b-53f1-4177-afb3-34cd9e4014ba None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:57.824216 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8529f87b-53f1-4177-afb3-34cd9e4014ba None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:57.825867 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 893/1766] 10.222.0.36 () {54 vars in 921 bytes} [Wed Aug 31 05:18:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:18:57.852661 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a076e3-d08f-45cf-bae3-759288ac4222 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:57.852661 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a076e3-d08f-45cf-bae3-759288ac4222 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:57.852925 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a076e3-d08f-45cf-bae3-759288ac4222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:57.881138 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-56a076e3-d08f-45cf-bae3-759288ac4222 None None] MFA Rules not processed for user `f27665b1574a4db7b7a9771f7d775826`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:57.882428 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-56a076e3-d08f-45cf-bae3-759288ac4222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:57.906479 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:58.003296 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 874/1767] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Aug 31 05:18:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:18:58.026549 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8529f87b-53f1-4177-afb3-34cd9e4014ba None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:58.029525 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8529f87b-53f1-4177-afb3-34cd9e4014ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:58.064450 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8529f87b-53f1-4177-afb3-34cd9e4014ba None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:58.084368 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c5668a-6a61-4b2b-b1e9-4b539519a446 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:58.085016 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5668a-6a61-4b2b-b1e9-4b539519a446 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:58.085298 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5668a-6a61-4b2b-b1e9-4b539519a446 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:58.085569 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5668a-6a61-4b2b-b1e9-4b539519a446 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:58.121320 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c5668a-6a61-4b2b-b1e9-4b539519a446 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:58.123043 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c5668a-6a61-4b2b-b1e9-4b539519a446 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:58.138351 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a076e3-d08f-45cf-bae3-759288ac4222 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:58.140584 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-56a076e3-d08f-45cf-bae3-759288ac4222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:58.174861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 894/1768] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:18:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:18:58.251282 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a076e3-d08f-45cf-bae3-759288ac4222 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:58.258239 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c5668a-6a61-4b2b-b1e9-4b539519a446 service nova] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:58.260407 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-95c5668a-6a61-4b2b-b1e9-4b539519a446 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:58.263699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c245a5-879f-404c-80ba-beb0a3563514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:58.264352 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c245a5-879f-404c-80ba-beb0a3563514 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:58.264628 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c245a5-879f-404c-80ba-beb0a3563514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:58.264891 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c245a5-879f-404c-80ba-beb0a3563514 admin admin] PATH_INFO: `/v3/projects/00e7a2da64bd4c29a237e87bff03c2e6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:58.266509 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c245a5-879f-404c-80ba-beb0a3563514 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00e7a2da64bd4c29a237e87bff03c2e6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:58.268837 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c245a5-879f-404c-80ba-beb0a3563514 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:58.284188 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c5668a-6a61-4b2b-b1e9-4b539519a446 service nova] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:58.297954 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b4c5b9-739e-420b-b602-4c3a421cad77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:58.298785 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4c5b9-739e-420b-b602-4c3a421cad77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:58.299129 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4c5b9-739e-420b-b602-4c3a421cad77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:58.299464 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4c5b9-739e-420b-b602-4c3a421cad77 admin admin] PATH_INFO: `/v3/projects/8ae84d5c59f0431887b06223cb120691` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:58.300684 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b4c5b9-739e-420b-b602-4c3a421cad77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ae84d5c59f0431887b06223cb120691)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:58.302864 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b4c5b9-739e-420b-b602-4c3a421cad77 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:58.338937 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-e3c245a5-879f-404c-80ba-beb0a3563514 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00e7a2da64bd4c29a237e87bff03c2e6', 'request_id': 'req-e3c245a5-879f-404c-80ba-beb0a3563514'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:58.347504 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-e3c245a5-879f-404c-80ba-beb0a3563514 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00e7a2da64bd4c29a237e87bff03c2e6', 'request_id': 'req-e3c245a5-879f-404c-80ba-beb0a3563514'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:58.357678 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-e3c245a5-879f-404c-80ba-beb0a3563514 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00e7a2da64bd4c29a237e87bff03c2e6', 'request_id': 'req-e3c245a5-879f-404c-80ba-beb0a3563514'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:58.373476 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a8b4c5b9-739e-420b-b602-4c3a421cad77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ae84d5c59f0431887b06223cb120691', 'request_id': 'req-a8b4c5b9-739e-420b-b602-4c3a421cad77'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:58.380217 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-e3c245a5-879f-404c-80ba-beb0a3563514 admin admin] The token cache is being invalidate because project 00e7a2da64bd4c29a237e87bff03c2e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:18:58.380690 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-e3c245a5-879f-404c-80ba-beb0a3563514 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:58.381446 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-e3c245a5-879f-404c-80ba-beb0a3563514 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:58.382632 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a8b4c5b9-739e-420b-b602-4c3a421cad77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ae84d5c59f0431887b06223cb120691', 'request_id': 'req-a8b4c5b9-739e-420b-b602-4c3a421cad77'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:58.387371 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 875/1769] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:18:58 2022] DELETE /identity/v3/projects/00e7a2da64bd4c29a237e87bff03c2e6 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:18:58.389241 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f615ccc-cebd-4cd1-bc9d-718ca5429df2 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:58.389598 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f615ccc-cebd-4cd1-bc9d-718ca5429df2 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:58.389933 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f615ccc-cebd-4cd1-bc9d-718ca5429df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:58.393142 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a8b4c5b9-739e-420b-b602-4c3a421cad77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ae84d5c59f0431887b06223cb120691', 'request_id': 'req-a8b4c5b9-739e-420b-b602-4c3a421cad77'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:58.407080 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a8b4c5b9-739e-420b-b602-4c3a421cad77 admin admin] The token cache is being invalidate because project 8ae84d5c59f0431887b06223cb120691 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:18:58.407362 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a8b4c5b9-739e-420b-b602-4c3a421cad77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:58.408432 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a8b4c5b9-739e-420b-b602-4c3a421cad77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:18:58.411393 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 895/1770] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:18:58 2022] DELETE /identity/v3/projects/8ae84d5c59f0431887b06223cb120691 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:58.415924 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-1f615ccc-cebd-4cd1-bc9d-718ca5429df2 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:58.417149 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-1f615ccc-cebd-4cd1-bc9d-718ca5429df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:58.419480 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb095fc-9725-4013-a1ea-896029a0694b None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:58.419831 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb095fc-9725-4013-a1ea-896029a0694b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:58.420164 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb095fc-9725-4013-a1ea-896029a0694b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:58.455486 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-afb095fc-9725-4013-a1ea-896029a0694b None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:58.459173 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-afb095fc-9725-4013-a1ea-896029a0694b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:58.479840 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:58.505907 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 876/1771] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:58.513594 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ad7b0-b3a5-43f6-8eaf-165b3b4b56fe None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:58.513992 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ad7b0-b3a5-43f6-8eaf-165b3b4b56fe None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:58.514357 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ad7b0-b3a5-43f6-8eaf-165b3b4b56fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:58.533848 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-8e0ad7b0-b3a5-43f6-8eaf-165b3b4b56fe None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:58.541306 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 896/1772] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:58.541448 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-8e0ad7b0-b3a5-43f6-8eaf-165b3b4b56fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:58.542612 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d06e7e-66bc-486b-9055-f34d5453fb94 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:58.542895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d06e7e-66bc-486b-9055-f34d5453fb94 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:58.543167 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d06e7e-66bc-486b-9055-f34d5453fb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:58.552492 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 877/1773] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:18:58.553324 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0ad7b0-b3a5-43f6-8eaf-165b3b4b56fe None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:58.563359 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0ad7b0-b3a5-43f6-8eaf-165b3b4b56fe None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:58.565033 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-94d06e7e-66bc-486b-9055-f34d5453fb94 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:58.566429 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-94d06e7e-66bc-486b-9055-f34d5453fb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:58.568157 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c21db1-5286-4840-8c41-b9564430b395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:58.568812 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c21db1-5286-4840-8c41-b9564430b395 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:58.569095 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c21db1-5286-4840-8c41-b9564430b395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:58.569369 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c21db1-5286-4840-8c41-b9564430b395 admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:58.570201 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c21db1-5286-4840-8c41-b9564430b395 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:18:58.570507 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c21db1-5286-4840-8c41-b9564430b395 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:58.572270 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c21db1-5286-4840-8c41-b9564430b395 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:58.575268 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 897/1774] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:18:58.579157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d06e7e-66bc-486b-9055-f34d5453fb94 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:58.591949 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 878/1775] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:18:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:18:58.592161 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d06e7e-66bc-486b-9055-f34d5453fb94 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:58.592935 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c21db1-5286-4840-8c41-b9564430b395 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:58.596963 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b34afba-a89a-4119-bc84-37d597b1f70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:58.597571 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34afba-a89a-4119-bc84-37d597b1f70d admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:58.597841 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34afba-a89a-4119-bc84-37d597b1f70d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:58.598138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34afba-a89a-4119-bc84-37d597b1f70d admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:58.598785 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b34afba-a89a-4119-bc84-37d597b1f70d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:18:58.599076 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b34afba-a89a-4119-bc84-37d597b1f70d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:58.601069 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b34afba-a89a-4119-bc84-37d597b1f70d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:58.605346 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c21db1-5286-4840-8c41-b9564430b395 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:58.611980 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903c1d0-d552-4a03-b97f-28d4af382a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:58.612833 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903c1d0-d552-4a03-b97f-28d4af382a7a admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:58.613180 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903c1d0-d552-4a03-b97f-28d4af382a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:58.613708 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903c1d0-d552-4a03-b97f-28d4af382a7a admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:58.615084 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903c1d0-d552-4a03-b97f-28d4af382a7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:58.617759 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 898/1776] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:18:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:18:58.621925 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903c1d0-d552-4a03-b97f-28d4af382a7a admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:58.623045 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b34afba-a89a-4119-bc84-37d597b1f70d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:58.630077 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:58.645134 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b34afba-a89a-4119-bc84-37d597b1f70d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:58.657253 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c785219-0842-458a-8b5b-af8643542691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:58.658663 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c785219-0842-458a-8b5b-af8643542691 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:58.659152 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c785219-0842-458a-8b5b-af8643542691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:58.660003 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c785219-0842-458a-8b5b-af8643542691 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:58.660505 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c785219-0842-458a-8b5b-af8643542691 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:58.662833 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c785219-0842-458a-8b5b-af8643542691 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:58.675635 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 879/1777] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:18:58 2022] POST /identity/v3/projects => generated 377 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:18:58.677785 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903c1d0-d552-4a03-b97f-28d4af382a7a admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:58.700560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 899/1778] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:18:58 2022] POST /identity/v3/projects => generated 365 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:18:58.705541 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c785219-0842-458a-8b5b-af8643542691 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:58.798582 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903c1d0-d552-4a03-b97f-28d4af382a7a admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:58.868571 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c785219-0842-458a-8b5b-af8643542691 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:58.878623 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef7743-75ca-4b8e-a334-97fc18fbdcb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:58.879478 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef7743-75ca-4b8e-a334-97fc18fbdcb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:58.879751 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef7743-75ca-4b8e-a334-97fc18fbdcb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:58.880022 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef7743-75ca-4b8e-a334-97fc18fbdcb1 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:58.882825 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ef7743-75ca-4b8e-a334-97fc18fbdcb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:58.889843 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ef7743-75ca-4b8e-a334-97fc18fbdcb1 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:58.912105 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b608c793-5e7a-455c-ab49-161cc83ddddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:58.912908 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608c793-5e7a-455c-ab49-161cc83ddddd admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:58.913401 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608c793-5e7a-455c-ab49-161cc83ddddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:58.913776 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608c793-5e7a-455c-ab49-161cc83ddddd admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:58.914625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b608c793-5e7a-455c-ab49-161cc83ddddd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:58.917011 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b608c793-5e7a-455c-ab49-161cc83ddddd admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:58.944700 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 880/1779] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:18:58 2022] POST /identity/v3/users => generated 289 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:18:58.950811 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 900/1780] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:18:58 2022] POST /identity/v3/users => generated 283 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:18:58.955010 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b608c793-5e7a-455c-ab49-161cc83ddddd admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:58.958804 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef7743-75ca-4b8e-a334-97fc18fbdcb1 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:58.972457 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef7743-75ca-4b8e-a334-97fc18fbdcb1 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:58.976176 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b608c793-5e7a-455c-ab49-161cc83ddddd admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:58.977431 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49d750-58e7-4019-802e-72baa146c3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:58.978232 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49d750-58e7-4019-802e-72baa146c3b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:58.978591 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49d750-58e7-4019-802e-72baa146c3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:58.978953 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49d750-58e7-4019-802e-72baa146c3b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:58.979690 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a49d750-58e7-4019-802e-72baa146c3b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:58.981463 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a49d750-58e7-4019-802e-72baa146c3b7 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:58.987085 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7170f5-43b0-4d5d-8f7e-1c80118c1727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:58.987918 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7170f5-43b0-4d5d-8f7e-1c80118c1727 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:58.988356 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7170f5-43b0-4d5d-8f7e-1c80118c1727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:58.988917 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7170f5-43b0-4d5d-8f7e-1c80118c1727 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:58.990330 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7170f5-43b0-4d5d-8f7e-1c80118c1727 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:58.992357 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7170f5-43b0-4d5d-8f7e-1c80118c1727 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:58.996843 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 901/1781] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:18:59.001182 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49d750-58e7-4019-802e-72baa146c3b7 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:59.012819 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 881/1782] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:18:59.014285 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7170f5-43b0-4d5d-8f7e-1c80118c1727 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:59.027875 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7170f5-43b0-4d5d-8f7e-1c80118c1727 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:59.029516 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49d750-58e7-4019-802e-72baa146c3b7 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:59.035047 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6792169-ad15-4866-8ae6-cdfd63df23ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:59.035688 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6792169-ad15-4866-8ae6-cdfd63df23ed admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:59.035964 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6792169-ad15-4866-8ae6-cdfd63df23ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:59.036229 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6792169-ad15-4866-8ae6-cdfd63df23ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:59.036868 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6792169-ad15-4866-8ae6-cdfd63df23ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:59.037841 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b2eb8-dd32-4995-823a-022ce325e259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:59.038675 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b2eb8-dd32-4995-823a-022ce325e259 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:59.040122 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6792169-ad15-4866-8ae6-cdfd63df23ed admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:59.040516 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b2eb8-dd32-4995-823a-022ce325e259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:59.041121 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b2eb8-dd32-4995-823a-022ce325e259 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:59.042981 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452b2eb8-dd32-4995-823a-022ce325e259 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:59.044858 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452b2eb8-dd32-4995-823a-022ce325e259 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:59.053186 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 902/1783] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:18:59.058430 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6792169-ad15-4866-8ae6-cdfd63df23ed admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:59.064294 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 882/1784] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:18:59.065673 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b2eb8-dd32-4995-823a-022ce325e259 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:59.071872 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6792169-ad15-4866-8ae6-cdfd63df23ed admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:59.076696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dafdb9f-b844-473f-a4cd-568724ff7686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:59.077460 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dafdb9f-b844-473f-a4cd-568724ff7686 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:59.077732 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dafdb9f-b844-473f-a4cd-568724ff7686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:59.078074 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dafdb9f-b844-473f-a4cd-568724ff7686 admin admin] PATH_INFO: `/v3/projects/f2b03d7d5e9b44c78fe3a776ddaf324a/users/c5d838479b2748a8854806f9991cd231/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:59.079520 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b2eb8-dd32-4995-823a-022ce325e259 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:59.084616 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b8d9ae-c22f-4434-aead-82288ac4e608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:59.085283 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8d9ae-c22f-4434-aead-82288ac4e608 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:59.085561 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8d9ae-c22f-4434-aead-82288ac4e608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:59.085832 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8d9ae-c22f-4434-aead-82288ac4e608 admin admin] PATH_INFO: `/v3/projects/40fec664b405481abeab6c40c31c5ef8/users/ba12eead104343819c2421428d35ea56/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:59.094825 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dafdb9f-b844-473f-a4cd-568724ff7686 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2b03d7d5e9b44c78fe3a776ddaf324a, user_id=c5d838479b2748a8854806f9991cd231, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:59.097549 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dafdb9f-b844-473f-a4cd-568724ff7686 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:59.098935 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:59.102520 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b8d9ae-c22f-4434-aead-82288ac4e608 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40fec664b405481abeab6c40c31c5ef8, user_id=ba12eead104343819c2421428d35ea56, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:59.106211 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b8d9ae-c22f-4434-aead-82288ac4e608 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:59.122119 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 903/1785] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:18:59 2022] PUT /identity/v3/projects/f2b03d7d5e9b44c78fe3a776ddaf324a/users/c5d838479b2748a8854806f9991cd231/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:59.122867 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 883/1786] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:18:59 2022] PUT /identity/v3/projects/40fec664b405481abeab6c40c31c5ef8/users/ba12eead104343819c2421428d35ea56/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:18:59.127337 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b993ffa-5903-4095-982e-fd2ebecfd2a9 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:59.127628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b993ffa-5903-4095-982e-fd2ebecfd2a9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:59.127902 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b993ffa-5903-4095-982e-fd2ebecfd2a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:59.133285 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c882e-fc70-4d66-b833-3906f5c64e36 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:59.133571 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c882e-fc70-4d66-b833-3906f5c64e36 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:59.133844 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c882e-fc70-4d66-b833-3906f5c64e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:59.165782 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-9b993ffa-5903-4095-982e-fd2ebecfd2a9 None None] MFA Rules not processed for user `c5d838479b2748a8854806f9991cd231`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:59.167057 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9b993ffa-5903-4095-982e-fd2ebecfd2a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:59.178805 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-0b0c882e-fc70-4d66-b833-3906f5c64e36 None None] MFA Rules not processed for user `ba12eead104343819c2421428d35ea56`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:59.180137 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-0b0c882e-fc70-4d66-b833-3906f5c64e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:59.259496 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 904/1787] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:18:59.263218 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b993ffa-5903-4095-982e-fd2ebecfd2a9 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:59.263362 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 884/1788] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:18:59.271213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0c882e-fc70-4d66-b833-3906f5c64e36 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:59.272736 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-0b0c882e-fc70-4d66-b833-3906f5c64e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:59.282378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:59.347045 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b993ffa-5903-4095-982e-fd2ebecfd2a9 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:59.352098 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5bc8fe-c9b2-441a-bbf6-31460a2d44f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:59.352723 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5bc8fe-c9b2-441a-bbf6-31460a2d44f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:59.352995 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5bc8fe-c9b2-441a-bbf6-31460a2d44f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:59.353257 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5bc8fe-c9b2-441a-bbf6-31460a2d44f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:59.353969 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5bc8fe-c9b2-441a-bbf6-31460a2d44f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:59.356223 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5bc8fe-c9b2-441a-bbf6-31460a2d44f9 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:59.367370 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0c882e-fc70-4d66-b833-3906f5c64e36 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:59.371295 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a2f943-e8d3-477b-917b-16450dacd48a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08e6024b0c8420aa274653c2a916edd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:59.371917 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2f943-e8d3-477b-917b-16450dacd48a service glance] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:59.372188 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2f943-e8d3-477b-917b-16450dacd48a service glance] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:59.372458 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2f943-e8d3-477b-917b-16450dacd48a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:59.388145 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a2f943-e8d3-477b-917b-16450dacd48a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:59.391676 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 905/1789] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:18:59 2022] POST /identity/v3/projects => generated 365 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:18:59.391830 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a2f943-e8d3-477b-917b-16450dacd48a service glance] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:59.392840 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5bc8fe-c9b2-441a-bbf6-31460a2d44f9 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:59.399686 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 885/1790] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:18:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:18:59.466882 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5bc8fe-c9b2-441a-bbf6-31460a2d44f9 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:59.472171 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc47292-deef-486b-ad80-ec280b248d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:59.472779 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc47292-deef-486b-ad80-ec280b248d50 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:59.473088 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc47292-deef-486b-ad80-ec280b248d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:59.473320 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc47292-deef-486b-ad80-ec280b248d50 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:59.473950 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc47292-deef-486b-ad80-ec280b248d50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:59.475943 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc47292-deef-486b-ad80-ec280b248d50 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:59.500539 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 906/1791] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:18:59 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:18:59.504968 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a2f943-e8d3-477b-917b-16450dacd48a service glance] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:59.515917 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a2f943-e8d3-477b-917b-16450dacd48a service glance] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:59.521209 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef72d7-c616-4cea-8bc6-fb2c23ecf191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:59.521864 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef72d7-c616-4cea-8bc6-fb2c23ecf191 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:59.522206 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef72d7-c616-4cea-8bc6-fb2c23ecf191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:59.522460 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef72d7-c616-4cea-8bc6-fb2c23ecf191 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:59.523153 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ef72d7-c616-4cea-8bc6-fb2c23ecf191 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:59.525022 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ef72d7-c616-4cea-8bc6-fb2c23ecf191 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:59.537044 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 886/1792] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:18:59.541453 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc47292-deef-486b-ad80-ec280b248d50 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:59.552919 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc47292-deef-486b-ad80-ec280b248d50 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:59.564280 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d916ef5-3ae9-4237-8140-e4ce4c654553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:59.564963 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d916ef5-3ae9-4237-8140-e4ce4c654553 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:59.565242 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d916ef5-3ae9-4237-8140-e4ce4c654553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:59.565508 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d916ef5-3ae9-4237-8140-e4ce4c654553 admin admin] PATH_INFO: `/v3/projects/6e70a213869f427db2da962cbd34b551/users/35162994690343ffa90326f764836a1e/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:59.580963 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d916ef5-3ae9-4237-8140-e4ce4c654553 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e70a213869f427db2da962cbd34b551, user_id=35162994690343ffa90326f764836a1e, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:59.583748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d916ef5-3ae9-4237-8140-e4ce4c654553 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:59.596892 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 907/1793] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:18:59 2022] PUT /identity/v3/projects/6e70a213869f427db2da962cbd34b551/users/35162994690343ffa90326f764836a1e/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:59.600982 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef72d7-c616-4cea-8bc6-fb2c23ecf191 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:59.611597 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef72d7-c616-4cea-8bc6-fb2c23ecf191 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:59.665123 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:59.675918 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf75579b-748e-4319-b3bd-c4537547bc4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:59.676524 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75579b-748e-4319-b3bd-c4537547bc4c admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:59.676797 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75579b-748e-4319-b3bd-c4537547bc4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:59.677305 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75579b-748e-4319-b3bd-c4537547bc4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:59.677997 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf75579b-748e-4319-b3bd-c4537547bc4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:59.679651 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf75579b-748e-4319-b3bd-c4537547bc4c admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:59.689559 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 887/1794] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:18:59.693959 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d916ef5-3ae9-4237-8140-e4ce4c654553 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:59.704955 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d916ef5-3ae9-4237-8140-e4ce4c654553 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:59.709679 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f6d0af-c3d5-4239-8387-86f7a86facb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:59.710419 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6d0af-c3d5-4239-8387-86f7a86facb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:59.710726 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6d0af-c3d5-4239-8387-86f7a86facb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:59.710930 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6d0af-c3d5-4239-8387-86f7a86facb7 admin admin] PATH_INFO: `/v3/projects/6e70a213869f427db2da962cbd34b551/users/35162994690343ffa90326f764836a1e/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:59.713178 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f6d0af-c3d5-4239-8387-86f7a86facb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e70a213869f427db2da962cbd34b551, user_id=35162994690343ffa90326f764836a1e, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:59.716161 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f6d0af-c3d5-4239-8387-86f7a86facb7 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:59.727417 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 908/1795] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:18:59 2022] PUT /identity/v3/projects/6e70a213869f427db2da962cbd34b551/users/35162994690343ffa90326f764836a1e/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:18:59.738859 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd4526-8e50-431e-9a45-5486d403d28e None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:59.739147 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd4526-8e50-431e-9a45-5486d403d28e None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:59.739417 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd4526-8e50-431e-9a45-5486d403d28e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:59.769366 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-46dd4526-8e50-431e-9a45-5486d403d28e None None] MFA Rules not processed for user `35162994690343ffa90326f764836a1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:18:59.770597 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-46dd4526-8e50-431e-9a45-5486d403d28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:59.854190 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 888/1796] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:18:59 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:18:59.869048 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f6d0af-c3d5-4239-8387-86f7a86facb7 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:18:59.871138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c4f6d0af-c3d5-4239-8387-86f7a86facb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:18:59.949309 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f6d0af-c3d5-4239-8387-86f7a86facb7 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:18:59.956192 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f57bb-ef9a-469f-8cef-dbffbfc8fefc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:18:59.956915 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f57bb-ef9a-469f-8cef-dbffbfc8fefc service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:18:59.958839 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f57bb-ef9a-469f-8cef-dbffbfc8fefc service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:18:59.958839 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f57bb-ef9a-469f-8cef-dbffbfc8fefc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:18:59.965303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:18:59.970584 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281f57bb-ef9a-469f-8cef-dbffbfc8fefc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:18:59.972095 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281f57bb-ef9a-469f-8cef-dbffbfc8fefc service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:18:59.985948 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 909/1797] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:18:59 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:19:00.069214 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd4526-8e50-431e-9a45-5486d403d28e None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:00.079207 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd4526-8e50-431e-9a45-5486d403d28e None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:00.082861 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f35f72f-a2e4-475f-a885-1937e2edf697 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:00.083660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f35f72f-a2e4-475f-a885-1937e2edf697 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:00.083945 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f35f72f-a2e4-475f-a885-1937e2edf697 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:00.084215 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f35f72f-a2e4-475f-a885-1937e2edf697 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:00.140730 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f35f72f-a2e4-475f-a885-1937e2edf697 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:00.142217 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f35f72f-a2e4-475f-a885-1937e2edf697 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:00.151025 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 889/1798] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:19:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:19:00.706951 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2428a2aa-0fad-44c4-a811-4f16c564d5ea None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:00.707313 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2428a2aa-0fad-44c4-a811-4f16c564d5ea None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:00.707675 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2428a2aa-0fad-44c4-a811-4f16c564d5ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:00.728770 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-2428a2aa-0fad-44c4-a811-4f16c564d5ea None None] MFA Rules not processed for user `09bf6db88ce8494a937211a814c9ff84`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:19:00.730379 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2428a2aa-0fad-44c4-a811-4f16c564d5ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:00.798827 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 910/1799] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Aug 31 05:19:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:19:00.812589 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f35f72f-a2e4-475f-a885-1937e2edf697 service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:00.814102 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-6f35f72f-a2e4-475f-a885-1937e2edf697 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:00.888012 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f35f72f-a2e4-475f-a885-1937e2edf697 service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:00.893186 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c9ac62-dae7-48ba-b034-5f9bcb347bf0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9b82080a73d4c4a8ec08b0e67d6d11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:00.893804 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c9ac62-dae7-48ba-b034-5f9bcb347bf0 service swift] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:00.894181 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c9ac62-dae7-48ba-b034-5f9bcb347bf0 service swift] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:00.894449 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c9ac62-dae7-48ba-b034-5f9bcb347bf0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:00.904025 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c9ac62-dae7-48ba-b034-5f9bcb347bf0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:00.905953 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c9ac62-dae7-48ba-b034-5f9bcb347bf0 service swift] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:00.919064 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 890/1800] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Aug 31 05:19:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 31 05:19:03.411511 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2428a2aa-0fad-44c4-a811-4f16c564d5ea None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:03.502249 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2428a2aa-0fad-44c4-a811-4f16c564d5ea None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:03.507129 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa481fc-d7d3-4d18-b47b-092349a91546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:03.507790 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa481fc-d7d3-4d18-b47b-092349a91546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:03.508103 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa481fc-d7d3-4d18-b47b-092349a91546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:03.508336 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa481fc-d7d3-4d18-b47b-092349a91546 admin admin] PATH_INFO: `/v3/users/c5d838479b2748a8854806f9991cd231` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:03.509512 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa481fc-d7d3-4d18-b47b-092349a91546 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5d838479b2748a8854806f9991cd231)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:03.511718 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa481fc-d7d3-4d18-b47b-092349a91546 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:03.578805 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5fa481fc-d7d3-4d18-b47b-092349a91546 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5d838479b2748a8854806f9991cd231'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:03.598158 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5fa481fc-d7d3-4d18-b47b-092349a91546 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5d838479b2748a8854806f9991cd231'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:03.617990 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5fa481fc-d7d3-4d18-b47b-092349a91546 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5d838479b2748a8854806f9991cd231'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:03.618779 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5fa481fc-d7d3-4d18-b47b-092349a91546 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5d838479b2748a8854806f9991cd231'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:03.644773 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5fa481fc-d7d3-4d18-b47b-092349a91546 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5d838479b2748a8854806f9991cd231'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:03.656756 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 911/1801] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:19:03 2022] DELETE /identity/v3/users/c5d838479b2748a8854806f9991cd231 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:19:03.658630 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a13e9c-697e-457a-a340-98d09f6b6276 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:03.659140 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a13e9c-697e-457a-a340-98d09f6b6276 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:03.659140 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a13e9c-697e-457a-a340-98d09f6b6276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:03.681559 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:03.701421 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-54a13e9c-697e-457a-a340-98d09f6b6276 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:19:03.703798 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-54a13e9c-697e-457a-a340-98d09f6b6276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:03.874192 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 891/1802] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:19:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:19:03.892994 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa481fc-d7d3-4d18-b47b-092349a91546 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:03.895411 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5fa481fc-d7d3-4d18-b47b-092349a91546 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:03.947219 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:04.010712 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa481fc-d7d3-4d18-b47b-092349a91546 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:04.015176 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4042e-8a0e-4639-a86d-51d1c0650540 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:04.016013 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4042e-8a0e-4639-a86d-51d1c0650540 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:04.016318 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4042e-8a0e-4639-a86d-51d1c0650540 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:04.016775 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4042e-8a0e-4639-a86d-51d1c0650540 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:04.036820 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb4042e-8a0e-4639-a86d-51d1c0650540 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:04.038810 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb4042e-8a0e-4639-a86d-51d1c0650540 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:04.065510 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 912/1803] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:19:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:19:04.479712 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a13e9c-697e-457a-a340-98d09f6b6276 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:04.481187 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-54a13e9c-697e-457a-a340-98d09f6b6276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:04.487910 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4042e-8a0e-4639-a86d-51d1c0650540 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:04.490556 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-afb4042e-8a0e-4639-a86d-51d1c0650540 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:04.502379 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a13e9c-697e-457a-a340-98d09f6b6276 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:04.514922 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98351337-2db2-450e-a4c9-6c5a09d1b393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:04.515620 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98351337-2db2-450e-a4c9-6c5a09d1b393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:04.515929 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98351337-2db2-450e-a4c9-6c5a09d1b393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:04.516148 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98351337-2db2-450e-a4c9-6c5a09d1b393 admin admin] PATH_INFO: `/v3/users/8e174a6bc62f4cf395d50a93725094e5` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:04.518549 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98351337-2db2-450e-a4c9-6c5a09d1b393 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e174a6bc62f4cf395d50a93725094e5)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:04.527037 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4042e-8a0e-4639-a86d-51d1c0650540 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:04.533854 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e030049-2feb-4d2c-95c8-0debe686b33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:04.534808 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e030049-2feb-4d2c-95c8-0debe686b33e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:04.535210 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e030049-2feb-4d2c-95c8-0debe686b33e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:04.535621 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e030049-2feb-4d2c-95c8-0debe686b33e admin admin] PATH_INFO: `/v3/projects/f2b03d7d5e9b44c78fe3a776ddaf324a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:04.537246 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e030049-2feb-4d2c-95c8-0debe686b33e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2b03d7d5e9b44c78fe3a776ddaf324a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:04.537926 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98351337-2db2-450e-a4c9-6c5a09d1b393 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:04.539334 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e030049-2feb-4d2c-95c8-0debe686b33e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:04.621201 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6e030049-2feb-4d2c-95c8-0debe686b33e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2b03d7d5e9b44c78fe3a776ddaf324a', 'request_id': 'req-6e030049-2feb-4d2c-95c8-0debe686b33e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:04.629361 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6e030049-2feb-4d2c-95c8-0debe686b33e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2b03d7d5e9b44c78fe3a776ddaf324a', 'request_id': 'req-6e030049-2feb-4d2c-95c8-0debe686b33e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:04.647699 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6e030049-2feb-4d2c-95c8-0debe686b33e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2b03d7d5e9b44c78fe3a776ddaf324a', 'request_id': 'req-6e030049-2feb-4d2c-95c8-0debe686b33e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:04.660416 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-98351337-2db2-450e-a4c9-6c5a09d1b393 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e174a6bc62f4cf395d50a93725094e5'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:04.670634 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6e030049-2feb-4d2c-95c8-0debe686b33e admin admin] The token cache is being invalidate because project f2b03d7d5e9b44c78fe3a776ddaf324a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:19:04.670890 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6e030049-2feb-4d2c-95c8-0debe686b33e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:04.671640 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6e030049-2feb-4d2c-95c8-0debe686b33e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:04.673601 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 913/1804] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:19:04 2022] DELETE /identity/v3/projects/f2b03d7d5e9b44c78fe3a776ddaf324a => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:19:04.678332 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e030049-2feb-4d2c-95c8-0debe686b33e admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:04.680166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6e030049-2feb-4d2c-95c8-0debe686b33e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:04.687875 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-98351337-2db2-450e-a4c9-6c5a09d1b393 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e174a6bc62f4cf395d50a93725094e5'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:04.688876 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-98351337-2db2-450e-a4c9-6c5a09d1b393 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e174a6bc62f4cf395d50a93725094e5'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:04.705224 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-98351337-2db2-450e-a4c9-6c5a09d1b393 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e174a6bc62f4cf395d50a93725094e5'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:04.707789 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-98351337-2db2-450e-a4c9-6c5a09d1b393 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e174a6bc62f4cf395d50a93725094e5'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:04.733591 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 892/1805] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:19:04 2022] DELETE /identity/v3/users/8e174a6bc62f4cf395d50a93725094e5 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:19:04.734908 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98351337-2db2-450e-a4c9-6c5a09d1b393 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:04.737064 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-98351337-2db2-450e-a4c9-6c5a09d1b393 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:04.825152 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98351337-2db2-450e-a4c9-6c5a09d1b393 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:04.828432 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e030049-2feb-4d2c-95c8-0debe686b33e admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:04.830897 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80573baa-2a8e-416b-891d-a9f0bec96bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:04.831821 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80573baa-2a8e-416b-891d-a9f0bec96bf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:04.832476 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80573baa-2a8e-416b-891d-a9f0bec96bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:04.832992 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80573baa-2a8e-416b-891d-a9f0bec96bf5 admin admin] PATH_INFO: `/v3/users/67b6a742d9e64a0e9894d33bd46dce2b` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:04.834394 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80573baa-2a8e-416b-891d-a9f0bec96bf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67b6a742d9e64a0e9894d33bd46dce2b)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:04.836548 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80573baa-2a8e-416b-891d-a9f0bec96bf5 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:04.838545 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259dd2b1-1be3-4991-9745-090a3e070e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:04.839342 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259dd2b1-1be3-4991-9745-090a3e070e14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:04.839714 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259dd2b1-1be3-4991-9745-090a3e070e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:04.839989 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259dd2b1-1be3-4991-9745-090a3e070e14 admin admin] PATH_INFO: `/v3/users/35162994690343ffa90326f764836a1e` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:04.841156 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259dd2b1-1be3-4991-9745-090a3e070e14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35162994690343ffa90326f764836a1e)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:04.843201 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259dd2b1-1be3-4991-9745-090a3e070e14 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:04.856253 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:04.918239 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-80573baa-2a8e-416b-891d-a9f0bec96bf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67b6a742d9e64a0e9894d33bd46dce2b'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:04.925192 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:04.927150 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-259dd2b1-1be3-4991-9745-090a3e070e14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35162994690343ffa90326f764836a1e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:04.939607 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-80573baa-2a8e-416b-891d-a9f0bec96bf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67b6a742d9e64a0e9894d33bd46dce2b'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:04.940412 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-80573baa-2a8e-416b-891d-a9f0bec96bf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67b6a742d9e64a0e9894d33bd46dce2b'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:04.941174 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-259dd2b1-1be3-4991-9745-090a3e070e14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35162994690343ffa90326f764836a1e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:04.950965 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-259dd2b1-1be3-4991-9745-090a3e070e14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35162994690343ffa90326f764836a1e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:04.952423 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-259dd2b1-1be3-4991-9745-090a3e070e14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35162994690343ffa90326f764836a1e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:04.953724 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-80573baa-2a8e-416b-891d-a9f0bec96bf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67b6a742d9e64a0e9894d33bd46dce2b'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:04.954800 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-80573baa-2a8e-416b-891d-a9f0bec96bf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67b6a742d9e64a0e9894d33bd46dce2b'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:04.966044 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-259dd2b1-1be3-4991-9745-090a3e070e14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35162994690343ffa90326f764836a1e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:04.976461 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 914/1806] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:19:04 2022] DELETE /identity/v3/users/35162994690343ffa90326f764836a1e => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:19:04.978318 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0dce2-07cb-41e6-83c4-0b00619cdbbf None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:04.978831 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0dce2-07cb-41e6-83c4-0b00619cdbbf None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:04.979266 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0dce2-07cb-41e6-83c4-0b00619cdbbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:05.002322 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-09f0dce2-07cb-41e6-83c4-0b00619cdbbf None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:19:05.003650 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-09f0dce2-07cb-41e6-83c4-0b00619cdbbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:05.016739 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 893/1807] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:19:04 2022] DELETE /identity/v3/users/67b6a742d9e64a0e9894d33bd46dce2b => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:19:05.018241 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7dccb3-5a69-427a-8e86-11e57c38ea56 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:05.018532 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7dccb3-5a69-427a-8e86-11e57c38ea56 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:05.018803 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7dccb3-5a69-427a-8e86-11e57c38ea56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:05.044033 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-dc7dccb3-5a69-427a-8e86-11e57c38ea56 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:19:05.045300 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-dc7dccb3-5a69-427a-8e86-11e57c38ea56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:05.126039 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 915/1808] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:19:05.138878 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0dce2-07cb-41e6-83c4-0b00619cdbbf None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:05.142605 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-09f0dce2-07cb-41e6-83c4-0b00619cdbbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:05.212468 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 894/1809] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:19:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:19:05.225568 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7dccb3-5a69-427a-8e86-11e57c38ea56 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:05.234200 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-dc7dccb3-5a69-427a-8e86-11e57c38ea56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:05.300097 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0dce2-07cb-41e6-83c4-0b00619cdbbf None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:05.400610 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7dccb3-5a69-427a-8e86-11e57c38ea56 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:05.415673 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c083095-1387-4dd3-998d-eabd34b65fc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:05.416620 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c083095-1387-4dd3-998d-eabd34b65fc7 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:05.416923 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c083095-1387-4dd3-998d-eabd34b65fc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:05.417184 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c083095-1387-4dd3-998d-eabd34b65fc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:05.451181 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c083095-1387-4dd3-998d-eabd34b65fc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:05.452684 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c083095-1387-4dd3-998d-eabd34b65fc7 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:05.466473 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 895/1810] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:19:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:19:05.486850 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db898114-7e2f-4a1c-8d93-5d677ce2be91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:05.491233 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db898114-7e2f-4a1c-8d93-5d677ce2be91 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:05.491906 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db898114-7e2f-4a1c-8d93-5d677ce2be91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:05.491906 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db898114-7e2f-4a1c-8d93-5d677ce2be91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:05.545451 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db898114-7e2f-4a1c-8d93-5d677ce2be91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:05.547305 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db898114-7e2f-4a1c-8d93-5d677ce2be91 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:05.556129 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c083095-1387-4dd3-998d-eabd34b65fc7 service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:05.557557 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-4c083095-1387-4dd3-998d-eabd34b65fc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:05.575231 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 916/1811] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:19:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 05:19:05.580053 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c083095-1387-4dd3-998d-eabd34b65fc7 service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:05.591490 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f5801-ae0e-40a6-8e8f-25f6f6065bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:05.593884 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f5801-ae0e-40a6-8e8f-25f6f6065bb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:05.594196 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f5801-ae0e-40a6-8e8f-25f6f6065bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:05.594516 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f5801-ae0e-40a6-8e8f-25f6f6065bb5 admin admin] PATH_INFO: `/v3/projects/6e70a213869f427db2da962cbd34b551` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:05.597057 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f5801-ae0e-40a6-8e8f-25f6f6065bb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e70a213869f427db2da962cbd34b551)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:05.599045 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f5801-ae0e-40a6-8e8f-25f6f6065bb5 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:05.691414 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:05.709196 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-e75f5801-ae0e-40a6-8e8f-25f6f6065bb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e70a213869f427db2da962cbd34b551', 'request_id': 'req-e75f5801-ae0e-40a6-8e8f-25f6f6065bb5'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:05.730524 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-e75f5801-ae0e-40a6-8e8f-25f6f6065bb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e70a213869f427db2da962cbd34b551', 'request_id': 'req-e75f5801-ae0e-40a6-8e8f-25f6f6065bb5'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:05.737958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-e75f5801-ae0e-40a6-8e8f-25f6f6065bb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e70a213869f427db2da962cbd34b551', 'request_id': 'req-e75f5801-ae0e-40a6-8e8f-25f6f6065bb5'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:05.761122 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-e75f5801-ae0e-40a6-8e8f-25f6f6065bb5 admin admin] The token cache is being invalidate because project 6e70a213869f427db2da962cbd34b551 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:19:05.761428 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-e75f5801-ae0e-40a6-8e8f-25f6f6065bb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:05.762110 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-e75f5801-ae0e-40a6-8e8f-25f6f6065bb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:05.764328 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 896/1812] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:19:05 2022] DELETE /identity/v3/projects/6e70a213869f427db2da962cbd34b551 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:19:05.769605 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a0b613-adf7-47d2-b42e-7b567de07d10 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:05.769952 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a0b613-adf7-47d2-b42e-7b567de07d10 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:05.770317 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a0b613-adf7-47d2-b42e-7b567de07d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:05.804117 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a0a0b613-adf7-47d2-b42e-7b567de07d10 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:19:05.804117 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a0a0b613-adf7-47d2-b42e-7b567de07d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:05.870184 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:06.035298 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 917/1813] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:19:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:19:06.036861 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d11560-80eb-4394-ac0d-f5ff55683aca None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:06.037408 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d11560-80eb-4394-ac0d-f5ff55683aca None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:06.037686 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d11560-80eb-4394-ac0d-f5ff55683aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:06.093798 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-a1d11560-80eb-4394-ac0d-f5ff55683aca None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:19:06.095608 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a1d11560-80eb-4394-ac0d-f5ff55683aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:06.109566 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 897/1814] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:19:06.114405 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a0b613-adf7-47d2-b42e-7b567de07d10 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:06.149620 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a0b613-adf7-47d2-b42e-7b567de07d10 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:06.174221 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0323bbc-ae68-48f8-8dbb-4981d2dc1650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:06.174975 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0323bbc-ae68-48f8-8dbb-4981d2dc1650 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:06.177114 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0323bbc-ae68-48f8-8dbb-4981d2dc1650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:06.177515 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0323bbc-ae68-48f8-8dbb-4981d2dc1650 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:06.178787 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0323bbc-ae68-48f8-8dbb-4981d2dc1650 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:19:06.179450 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0323bbc-ae68-48f8-8dbb-4981d2dc1650 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:06.183057 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0323bbc-ae68-48f8-8dbb-4981d2dc1650 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:06.222079 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 918/1815] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:19:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:19:06.223324 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d11560-80eb-4394-ac0d-f5ff55683aca None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:06.247309 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d11560-80eb-4394-ac0d-f5ff55683aca None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:06.256222 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cc57d0-11b2-44fc-8b8c-69e3581ca3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:06.256831 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc57d0-11b2-44fc-8b8c-69e3581ca3f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:06.257112 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc57d0-11b2-44fc-8b8c-69e3581ca3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:06.257383 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc57d0-11b2-44fc-8b8c-69e3581ca3f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:06.258449 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cc57d0-11b2-44fc-8b8c-69e3581ca3f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:06.260658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cc57d0-11b2-44fc-8b8c-69e3581ca3f9 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:06.297404 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0323bbc-ae68-48f8-8dbb-4981d2dc1650 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:06.299179 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a0323bbc-ae68-48f8-8dbb-4981d2dc1650 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:06.319744 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0323bbc-ae68-48f8-8dbb-4981d2dc1650 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:06.325566 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770dacd4-5fb1-49ab-a50d-730ba9d574bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:06.326400 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770dacd4-5fb1-49ab-a50d-730ba9d574bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:06.326770 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770dacd4-5fb1-49ab-a50d-730ba9d574bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:06.327138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770dacd4-5fb1-49ab-a50d-730ba9d574bd admin admin] PATH_INFO: `/v3/projects/ad793b5517824686abbab84a493d792e` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:06.328732 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770dacd4-5fb1-49ab-a50d-730ba9d574bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad793b5517824686abbab84a493d792e)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:06.331978 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770dacd4-5fb1-49ab-a50d-730ba9d574bd admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:06.356760 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 898/1816] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:19:06 2022] POST /identity/v3/projects => generated 375 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:19:06.358182 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cc57d0-11b2-44fc-8b8c-69e3581ca3f9 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:06.431577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-770dacd4-5fb1-49ab-a50d-730ba9d574bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad793b5517824686abbab84a493d792e', 'request_id': 'req-770dacd4-5fb1-49ab-a50d-730ba9d574bd'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:06.449521 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-770dacd4-5fb1-49ab-a50d-730ba9d574bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad793b5517824686abbab84a493d792e', 'request_id': 'req-770dacd4-5fb1-49ab-a50d-730ba9d574bd'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:06.461543 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-770dacd4-5fb1-49ab-a50d-730ba9d574bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad793b5517824686abbab84a493d792e', 'request_id': 'req-770dacd4-5fb1-49ab-a50d-730ba9d574bd'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:06.496808 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-770dacd4-5fb1-49ab-a50d-730ba9d574bd admin admin] The token cache is being invalidate because project ad793b5517824686abbab84a493d792e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:19:06.496808 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-770dacd4-5fb1-49ab-a50d-730ba9d574bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:06.496808 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-770dacd4-5fb1-49ab-a50d-730ba9d574bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:06.496808 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 919/1817] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:19:06 2022] DELETE /identity/v3/projects/ad793b5517824686abbab84a493d792e => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:19:06.500050 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770dacd4-5fb1-49ab-a50d-730ba9d574bd admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:06.503483 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-770dacd4-5fb1-49ab-a50d-730ba9d574bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:06.559250 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cc57d0-11b2-44fc-8b8c-69e3581ca3f9 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:06.601887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770dacd4-5fb1-49ab-a50d-730ba9d574bd admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:06.615472 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04d4fae-ec66-423c-81ef-32cf286e99cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:06.616473 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d4fae-ec66-423c-81ef-32cf286e99cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:06.616930 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d4fae-ec66-423c-81ef-32cf286e99cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:06.617374 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d4fae-ec66-423c-81ef-32cf286e99cd admin admin] PATH_INFO: `/v3/users/6c909fdb62f04edfb9e4953b24ee815f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:06.618838 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04d4fae-ec66-423c-81ef-32cf286e99cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c909fdb62f04edfb9e4953b24ee815f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:06.621871 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04d4fae-ec66-423c-81ef-32cf286e99cd admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:06.707811 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baeea63-d86d-40d3-8977-4e44fcc79d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:06.708444 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baeea63-d86d-40d3-8977-4e44fcc79d18 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:06.708864 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baeea63-d86d-40d3-8977-4e44fcc79d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:06.709165 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baeea63-d86d-40d3-8977-4e44fcc79d18 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:06.709790 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baeea63-d86d-40d3-8977-4e44fcc79d18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:06.717671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:06.718859 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baeea63-d86d-40d3-8977-4e44fcc79d18 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:06.736888 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e04d4fae-ec66-423c-81ef-32cf286e99cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c909fdb62f04edfb9e4953b24ee815f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:06.749293 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e04d4fae-ec66-423c-81ef-32cf286e99cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c909fdb62f04edfb9e4953b24ee815f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:06.765775 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e04d4fae-ec66-423c-81ef-32cf286e99cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c909fdb62f04edfb9e4953b24ee815f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:06.767279 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e04d4fae-ec66-423c-81ef-32cf286e99cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c909fdb62f04edfb9e4953b24ee815f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:06.772263 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 899/1818] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:19:06 2022] POST /identity/v3/users => generated 288 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:19:06.776748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baeea63-d86d-40d3-8977-4e44fcc79d18 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:06.780804 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-1baeea63-d86d-40d3-8977-4e44fcc79d18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:06.798579 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e04d4fae-ec66-423c-81ef-32cf286e99cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c909fdb62f04edfb9e4953b24ee815f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:06.805880 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baeea63-d86d-40d3-8977-4e44fcc79d18 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:06.811464 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 920/1819] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:19:06 2022] DELETE /identity/v3/users/6c909fdb62f04edfb9e4953b24ee815f => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:19:06.813222 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568883e9-868b-4b45-afc1-3edb129b64a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:06.813914 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568883e9-868b-4b45-afc1-3edb129b64a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:06.814295 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568883e9-868b-4b45-afc1-3edb129b64a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:06.814926 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568883e9-868b-4b45-afc1-3edb129b64a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:06.815674 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568883e9-868b-4b45-afc1-3edb129b64a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:06.817933 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568883e9-868b-4b45-afc1-3edb129b64a3 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:06.834047 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 900/1820] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:19:06.837531 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04d4fae-ec66-423c-81ef-32cf286e99cd admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:06.839231 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e04d4fae-ec66-423c-81ef-32cf286e99cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:06.958955 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04d4fae-ec66-423c-81ef-32cf286e99cd admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:06.964486 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03b10b-3702-4efa-8a65-54fcf96df425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:06.965862 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03b10b-3702-4efa-8a65-54fcf96df425 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:06.968109 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03b10b-3702-4efa-8a65-54fcf96df425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:06.968449 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03b10b-3702-4efa-8a65-54fcf96df425 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:06.969156 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f03b10b-3702-4efa-8a65-54fcf96df425 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:06.970869 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f03b10b-3702-4efa-8a65-54fcf96df425 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:06.985179 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 921/1821] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:19:06.990255 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568883e9-868b-4b45-afc1-3edb129b64a3 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:07.003845 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568883e9-868b-4b45-afc1-3edb129b64a3 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:07.011685 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb3a16-f252-474c-bbc4-d3ae02d0e69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:07.012531 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb3a16-f252-474c-bbc4-d3ae02d0e69b admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:07.012905 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb3a16-f252-474c-bbc4-d3ae02d0e69b admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:07.013295 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb3a16-f252-474c-bbc4-d3ae02d0e69b admin admin] PATH_INFO: `/v3/projects/a12bfd6a0f9d4b559a0c2f5f26af4933/users/3435046783404393948a40d2d61043a3/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:07.042945 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bb3a16-f252-474c-bbc4-d3ae02d0e69b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a12bfd6a0f9d4b559a0c2f5f26af4933, user_id=3435046783404393948a40d2d61043a3, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:07.045472 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bb3a16-f252-474c-bbc4-d3ae02d0e69b admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:07.048856 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:07.072136 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 901/1822] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:19:06 2022] PUT /identity/v3/projects/a12bfd6a0f9d4b559a0c2f5f26af4933/users/3435046783404393948a40d2d61043a3/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:19:07.078833 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e604c-fff3-4309-b541-5f9d2daa1aab None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:07.079245 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e604c-fff3-4309-b541-5f9d2daa1aab None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:07.079638 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e604c-fff3-4309-b541-5f9d2daa1aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:07.130397 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-7f5e604c-fff3-4309-b541-5f9d2daa1aab None None] MFA Rules not processed for user `3435046783404393948a40d2d61043a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:19:07.131712 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7f5e604c-fff3-4309-b541-5f9d2daa1aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:07.233668 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb3a16-f252-474c-bbc4-d3ae02d0e69b admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:07.236335 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-09bb3a16-f252-474c-bbc4-d3ae02d0e69b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:07.270663 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 922/1823] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:19:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:19:07.275096 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e604c-fff3-4309-b541-5f9d2daa1aab None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:07.374260 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:07.378408 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb3a16-f252-474c-bbc4-d3ae02d0e69b admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:07.386678 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d49b3a-b7c1-4179-9222-682f9f871c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:07.387868 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d49b3a-b7c1-4179-9222-682f9f871c2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:07.388395 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d49b3a-b7c1-4179-9222-682f9f871c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:07.388874 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d49b3a-b7c1-4179-9222-682f9f871c2b admin admin] PATH_INFO: `/v3/projects/2fd6349fbb4448eabd9d179580d41718` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:07.390443 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d49b3a-b7c1-4179-9222-682f9f871c2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fd6349fbb4448eabd9d179580d41718)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:07.392765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d49b3a-b7c1-4179-9222-682f9f871c2b admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:07.405654 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e604c-fff3-4309-b541-5f9d2daa1aab None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:07.412086 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e477ddb-a2a1-4f74-a111-01b9351cd7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:07.412724 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e477ddb-a2a1-4f74-a111-01b9351cd7c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:07.413001 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e477ddb-a2a1-4f74-a111-01b9351cd7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:07.426045 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e477ddb-a2a1-4f74-a111-01b9351cd7c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:07.426833 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e477ddb-a2a1-4f74-a111-01b9351cd7c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:07.430340 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e477ddb-a2a1-4f74-a111-01b9351cd7c8 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:07.461007 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-93d49b3a-b7c1-4179-9222-682f9f871c2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fd6349fbb4448eabd9d179580d41718', 'request_id': 'req-93d49b3a-b7c1-4179-9222-682f9f871c2b'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:07.467841 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 923/1824] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:19:07 2022] POST /identity/v3/projects => generated 377 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:19:07.468786 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e477ddb-a2a1-4f74-a111-01b9351cd7c8 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:07.470435 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1e477ddb-a2a1-4f74-a111-01b9351cd7c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:07.470932 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-93d49b3a-b7c1-4179-9222-682f9f871c2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fd6349fbb4448eabd9d179580d41718', 'request_id': 'req-93d49b3a-b7c1-4179-9222-682f9f871c2b'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:07.479919 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-93d49b3a-b7c1-4179-9222-682f9f871c2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fd6349fbb4448eabd9d179580d41718', 'request_id': 'req-93d49b3a-b7c1-4179-9222-682f9f871c2b'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:07.533838 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-93d49b3a-b7c1-4179-9222-682f9f871c2b admin admin] The token cache is being invalidate because project 2fd6349fbb4448eabd9d179580d41718 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:19:07.534738 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-93d49b3a-b7c1-4179-9222-682f9f871c2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:07.537028 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-93d49b3a-b7c1-4179-9222-682f9f871c2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:07.538930 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 902/1825] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:19:07 2022] DELETE /identity/v3/projects/2fd6349fbb4448eabd9d179580d41718 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:19:07.539431 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d49b3a-b7c1-4179-9222-682f9f871c2b admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:07.546616 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-93d49b3a-b7c1-4179-9222-682f9f871c2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:07.566375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e477ddb-a2a1-4f74-a111-01b9351cd7c8 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:07.573091 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc01ea60-1aef-4fe5-a635-541f504733b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:07.573986 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc01ea60-1aef-4fe5-a635-541f504733b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:07.574296 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc01ea60-1aef-4fe5-a635-541f504733b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:07.574571 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc01ea60-1aef-4fe5-a635-541f504733b2 admin admin] PATH_INFO: `/v3/projects/b92519408725413db28b3d593ff63a1f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:07.576365 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc01ea60-1aef-4fe5-a635-541f504733b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b92519408725413db28b3d593ff63a1f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:07.578617 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc01ea60-1aef-4fe5-a635-541f504733b2 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:07.695064 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dc01ea60-1aef-4fe5-a635-541f504733b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b92519408725413db28b3d593ff63a1f', 'request_id': 'req-dc01ea60-1aef-4fe5-a635-541f504733b2'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:07.697140 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d49b3a-b7c1-4179-9222-682f9f871c2b admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:07.706651 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dc01ea60-1aef-4fe5-a635-541f504733b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b92519408725413db28b3d593ff63a1f', 'request_id': 'req-dc01ea60-1aef-4fe5-a635-541f504733b2'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:07.721258 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dc01ea60-1aef-4fe5-a635-541f504733b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b92519408725413db28b3d593ff63a1f', 'request_id': 'req-dc01ea60-1aef-4fe5-a635-541f504733b2'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:07.739243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dc01ea60-1aef-4fe5-a635-541f504733b2 admin admin] The token cache is being invalidate because project b92519408725413db28b3d593ff63a1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:19:07.739696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dc01ea60-1aef-4fe5-a635-541f504733b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:07.740535 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dc01ea60-1aef-4fe5-a635-541f504733b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:07.742475 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 924/1826] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:19:07 2022] DELETE /identity/v3/projects/b92519408725413db28b3d593ff63a1f => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:19:07.743504 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262e4dd-6229-4234-872a-166e2aebfdfa None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:07.743899 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262e4dd-6229-4234-872a-166e2aebfdfa None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:07.744261 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262e4dd-6229-4234-872a-166e2aebfdfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:07.765793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:07.768788 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f262e4dd-6229-4234-872a-166e2aebfdfa None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:19:07.770395 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f262e4dd-6229-4234-872a-166e2aebfdfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:07.810030 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de5eb86-44ad-4805-b6c7-4ee8e0d72b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:07.810030 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de5eb86-44ad-4805-b6c7-4ee8e0d72b49 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:07.810030 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de5eb86-44ad-4805-b6c7-4ee8e0d72b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:07.810030 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de5eb86-44ad-4805-b6c7-4ee8e0d72b49 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:07.810623 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de5eb86-44ad-4805-b6c7-4ee8e0d72b49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:07.810623 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de5eb86-44ad-4805-b6c7-4ee8e0d72b49 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:07.834204 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 903/1827] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:19:07 2022] POST /identity/v3/users => generated 295 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:19:07.836064 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d952207-9311-4d66-a8ee-7a9d40f624b1 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:07.836064 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d952207-9311-4d66-a8ee-7a9d40f624b1 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:07.836064 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d952207-9311-4d66-a8ee-7a9d40f624b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:07.845700 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:07.857922 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-6d952207-9311-4d66-a8ee-7a9d40f624b1 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:19:07.859461 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-6d952207-9311-4d66-a8ee-7a9d40f624b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:07.862421 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 925/1828] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:19:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:19:07.863121 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f262e4dd-6229-4234-872a-166e2aebfdfa None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:07.864360 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f262e4dd-6229-4234-872a-166e2aebfdfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:07.868377 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 904/1829] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:19:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:19:07.871434 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e29a10-cbbd-4c8e-b1f5-05194f79b95c None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:07.871729 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e29a10-cbbd-4c8e-b1f5-05194f79b95c None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:07.872004 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e29a10-cbbd-4c8e-b1f5-05194f79b95c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:07.879441 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f262e4dd-6229-4234-872a-166e2aebfdfa None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:07.884250 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb08f8-6a37-45b6-bf2e-a25ce12e8d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:07.884841 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb08f8-6a37-45b6-bf2e-a25ce12e8d58 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:07.885119 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb08f8-6a37-45b6-bf2e-a25ce12e8d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:07.885385 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb08f8-6a37-45b6-bf2e-a25ce12e8d58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:07.886061 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eb08f8-6a37-45b6-bf2e-a25ce12e8d58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:07.887647 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eb08f8-6a37-45b6-bf2e-a25ce12e8d58 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:07.893851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-47e29a10-cbbd-4c8e-b1f5-05194f79b95c None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:19:07.895203 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-47e29a10-cbbd-4c8e-b1f5-05194f79b95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:07.897839 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 926/1830] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:19:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:19:07.899101 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86eb12-700e-401b-9e0c-5f6cb5390788 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:07.899387 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86eb12-700e-401b-9e0c-5f6cb5390788 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:07.899660 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86eb12-700e-401b-9e0c-5f6cb5390788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:07.904261 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 905/1831] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:19:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:19:07.905110 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e29a10-cbbd-4c8e-b1f5-05194f79b95c None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:07.915200 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e29a10-cbbd-4c8e-b1f5-05194f79b95c None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:07.920223 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-5d86eb12-700e-401b-9e0c-5f6cb5390788 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:19:07.920507 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df4b60-4281-48ea-9477-87efbc8c1772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:07.921234 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df4b60-4281-48ea-9477-87efbc8c1772 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:07.921557 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5d86eb12-700e-401b-9e0c-5f6cb5390788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:07.921676 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df4b60-4281-48ea-9477-87efbc8c1772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:07.922029 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df4b60-4281-48ea-9477-87efbc8c1772 admin admin] PATH_INFO: `/v3/projects/d733981362f94187826314d8d9829774/users/e5322876703b4d73b7cae1e7bf39b1b4/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:07.932727 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 927/1832] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:19:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:19:07.932727 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d86eb12-700e-401b-9e0c-5f6cb5390788 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:07.940064 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d86eb12-700e-401b-9e0c-5f6cb5390788 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:07.940356 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0df4b60-4281-48ea-9477-87efbc8c1772 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d733981362f94187826314d8d9829774, user_id=e5322876703b4d73b7cae1e7bf39b1b4, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:07.943088 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0df4b60-4281-48ea-9477-87efbc8c1772 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:07.945477 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698448e1-d210-47fb-94da-289359c80a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:07.946720 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698448e1-d210-47fb-94da-289359c80a5f admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:07.947005 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698448e1-d210-47fb-94da-289359c80a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:07.947385 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698448e1-d210-47fb-94da-289359c80a5f admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:07.948026 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698448e1-d210-47fb-94da-289359c80a5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:19:07.948323 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698448e1-d210-47fb-94da-289359c80a5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:07.949916 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698448e1-d210-47fb-94da-289359c80a5f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:07.959826 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 906/1833] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:19:07 2022] PUT /identity/v3/projects/d733981362f94187826314d8d9829774/users/e5322876703b4d73b7cae1e7bf39b1b4/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:19:07.960907 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df4b60-4281-48ea-9477-87efbc8c1772 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:07.967012 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 928/1834] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:19:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:19:07.968525 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b1e4c-69b8-498a-9bf8-5cd90840fa4b None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:07.968807 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b1e4c-69b8-498a-9bf8-5cd90840fa4b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:07.969076 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b1e4c-69b8-498a-9bf8-5cd90840fa4b None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:07.970913 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 929/1835] 10.222.0.36 () {54 vars in 921 bytes} [Wed Aug 31 05:19:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:19:07.971749 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b1e4c-69b8-498a-9bf8-5cd90840fa4b None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:08.038446 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df4b60-4281-48ea-9477-87efbc8c1772 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:08.043223 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9249e9d2-80be-4329-ad56-779e23213928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:08.043850 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9249e9d2-80be-4329-ad56-779e23213928 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:08.044087 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b1e4c-69b8-498a-9bf8-5cd90840fa4b None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:08.044187 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9249e9d2-80be-4329-ad56-779e23213928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:08.044411 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9249e9d2-80be-4329-ad56-779e23213928 admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:08.045096 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9249e9d2-80be-4329-ad56-779e23213928 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:19:08.045462 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9249e9d2-80be-4329-ad56-779e23213928 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:08.047607 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9249e9d2-80be-4329-ad56-779e23213928 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:08.053825 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac4c632-82ab-4e5e-8ea9-c785c95fff08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:08.056983 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac4c632-82ab-4e5e-8ea9-c785c95fff08 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:08.057562 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac4c632-82ab-4e5e-8ea9-c785c95fff08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:08.057834 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac4c632-82ab-4e5e-8ea9-c785c95fff08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:08.058548 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac4c632-82ab-4e5e-8ea9-c785c95fff08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:08.060172 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac4c632-82ab-4e5e-8ea9-c785c95fff08 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:08.061334 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 907/1836] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:19:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:19:08.062436 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9249e9d2-80be-4329-ad56-779e23213928 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:08.073031 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9249e9d2-80be-4329-ad56-779e23213928 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:08.077102 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 930/1837] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:19:07 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:19:08.077927 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14be19b5-7a98-4385-9fe0-5c9d905b33cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:08.078279 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37687a99-9a89-4341-9052-c38e3935dab2 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:08.078559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37687a99-9a89-4341-9052-c38e3935dab2 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:08.078715 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be19b5-7a98-4385-9fe0-5c9d905b33cc admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:08.078815 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37687a99-9a89-4341-9052-c38e3935dab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:08.079141 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be19b5-7a98-4385-9fe0-5c9d905b33cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:08.079486 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be19b5-7a98-4385-9fe0-5c9d905b33cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:08.080237 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14be19b5-7a98-4385-9fe0-5c9d905b33cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:08.082395 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14be19b5-7a98-4385-9fe0-5c9d905b33cc admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:08.099365 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-37687a99-9a89-4341-9052-c38e3935dab2 None None] MFA Rules not processed for user `f27665b1574a4db7b7a9771f7d775826`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:19:08.100464 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-37687a99-9a89-4341-9052-c38e3935dab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:08.117567 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 908/1838] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:19:08 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:19:08.117567 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14be19b5-7a98-4385-9fe0-5c9d905b33cc admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:08.170981 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 931/1839] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Aug 31 05:19:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:19:08.172123 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37687a99-9a89-4341-9052-c38e3935dab2 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:08.177294 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:08.182768 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37687a99-9a89-4341-9052-c38e3935dab2 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:08.185135 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14be19b5-7a98-4385-9fe0-5c9d905b33cc admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:08.189605 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc8f1f1-d2c4-4b66-bddb-e8750994a9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:08.190313 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8f1f1-d2c4-4b66-bddb-e8750994a9b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:08.190595 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8f1f1-d2c4-4b66-bddb-e8750994a9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:08.190872 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8f1f1-d2c4-4b66-bddb-e8750994a9b9 admin admin] PATH_INFO: `/v3/projects/d733981362f94187826314d8d9829774/users/e5322876703b4d73b7cae1e7bf39b1b4/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:08.192610 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d9ec86-102c-47e4-a8ad-b78893cf06d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:08.192827 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc8f1f1-d2c4-4b66-bddb-e8750994a9b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d733981362f94187826314d8d9829774, user_id=e5322876703b4d73b7cae1e7bf39b1b4, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:08.193692 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9ec86-102c-47e4-a8ad-b78893cf06d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:08.194184 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9ec86-102c-47e4-a8ad-b78893cf06d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:08.194647 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9ec86-102c-47e4-a8ad-b78893cf06d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:08.195463 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d9ec86-102c-47e4-a8ad-b78893cf06d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:08.195666 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc8f1f1-d2c4-4b66-bddb-e8750994a9b9 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:08.197794 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d9ec86-102c-47e4-a8ad-b78893cf06d3 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:08.209736 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 932/1840] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:19:08 2022] PUT /identity/v3/projects/d733981362f94187826314d8d9829774/users/e5322876703b4d73b7cae1e7bf39b1b4/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:19:08.219202 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc8f1f1-d2c4-4b66-bddb-e8750994a9b9 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:08.227787 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 909/1841] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:19:08 2022] POST /identity/v3/projects => generated 373 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:19:08.228556 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d9ec86-102c-47e4-a8ad-b78893cf06d3 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:08.232737 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-67d9ec86-102c-47e4-a8ad-b78893cf06d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:08.270325 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:08.293561 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc8f1f1-d2c4-4b66-bddb-e8750994a9b9 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:08.312892 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d9ec86-102c-47e4-a8ad-b78893cf06d3 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:08.318648 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf32715e-dd2a-456b-9370-89daa2e8b388 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:08.319258 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32715e-dd2a-456b-9370-89daa2e8b388 service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:08.319536 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32715e-dd2a-456b-9370-89daa2e8b388 service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:08.319806 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32715e-dd2a-456b-9370-89daa2e8b388 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:08.329627 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf32715e-dd2a-456b-9370-89daa2e8b388 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:08.331325 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf32715e-dd2a-456b-9370-89daa2e8b388 service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:08.353971 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 910/1842] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:19:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:19:08.355480 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d17e1-5bb6-4e63-b1f7-41f69197076c None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:08.355793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d17e1-5bb6-4e63-b1f7-41f69197076c None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:08.356066 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d17e1-5bb6-4e63-b1f7-41f69197076c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:08.366864 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840908f9-5380-4df2-aad5-25c46a6dd4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:08.367557 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840908f9-5380-4df2-aad5-25c46a6dd4e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:08.367956 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840908f9-5380-4df2-aad5-25c46a6dd4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:08.368299 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840908f9-5380-4df2-aad5-25c46a6dd4e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:08.369003 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840908f9-5380-4df2-aad5-25c46a6dd4e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:08.371481 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840908f9-5380-4df2-aad5-25c46a6dd4e9 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:08.392919 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-300d17e1-5bb6-4e63-b1f7-41f69197076c None None] MFA Rules not processed for user `e5322876703b4d73b7cae1e7bf39b1b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:19:08.394391 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-300d17e1-5bb6-4e63-b1f7-41f69197076c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:08.404956 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 933/1843] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:19:08 2022] POST /identity/v3/users => generated 290 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:19:08.405930 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840908f9-5380-4df2-aad5-25c46a6dd4e9 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:08.419902 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840908f9-5380-4df2-aad5-25c46a6dd4e9 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:08.425573 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3041f-b934-4397-882d-0e53eb7c3d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:08.426222 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3041f-b934-4397-882d-0e53eb7c3d42 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:08.426500 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3041f-b934-4397-882d-0e53eb7c3d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:08.426767 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3041f-b934-4397-882d-0e53eb7c3d42 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:08.428297 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a3041f-b934-4397-882d-0e53eb7c3d42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:08.430920 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a3041f-b934-4397-882d-0e53eb7c3d42 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:08.461592 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 934/1844] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:19:08 2022] POST /identity/v3/users => generated 287 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:19:08.462347 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3041f-b934-4397-882d-0e53eb7c3d42 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:08.477599 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3041f-b934-4397-882d-0e53eb7c3d42 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:08.482489 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7fa1ca-d2de-4337-af35-3a0537472622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:08.483153 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7fa1ca-d2de-4337-af35-3a0537472622 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:08.483416 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7fa1ca-d2de-4337-af35-3a0537472622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:08.483731 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7fa1ca-d2de-4337-af35-3a0537472622 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:08.484423 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7fa1ca-d2de-4337-af35-3a0537472622 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:08.487026 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7fa1ca-d2de-4337-af35-3a0537472622 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:08.493200 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 911/1845] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:19:08 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:19:08.494197 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d17e1-5bb6-4e63-b1f7-41f69197076c None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:08.502514 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 935/1846] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:19:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:19:08.504786 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7fa1ca-d2de-4337-af35-3a0537472622 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:08.509007 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d17e1-5bb6-4e63-b1f7-41f69197076c None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:08.514131 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3424482f-f6fe-4eff-8096-615540b3f1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:08.514742 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424482f-f6fe-4eff-8096-615540b3f1ec admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:08.515017 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424482f-f6fe-4eff-8096-615540b3f1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:08.515317 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424482f-f6fe-4eff-8096-615540b3f1ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:08.515958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3424482f-f6fe-4eff-8096-615540b3f1ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:08.516270 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7fa1ca-d2de-4337-af35-3a0537472622 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:08.517562 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3424482f-f6fe-4eff-8096-615540b3f1ec admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:08.522847 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01a8e2-58aa-4f4d-96c7-6bae4260bb2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:08.524112 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01a8e2-58aa-4f4d-96c7-6bae4260bb2d admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:08.524398 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01a8e2-58aa-4f4d-96c7-6bae4260bb2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:08.524668 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01a8e2-58aa-4f4d-96c7-6bae4260bb2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:08.526411 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c01a8e2-58aa-4f4d-96c7-6bae4260bb2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:08.528043 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c01a8e2-58aa-4f4d-96c7-6bae4260bb2d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:08.531586 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 912/1847] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:19:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:19:08.531586 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3424482f-f6fe-4eff-8096-615540b3f1ec admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:08.532636 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3424482f-f6fe-4eff-8096-615540b3f1ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:08.543412 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 936/1848] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:19:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:19:08.543412 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01a8e2-58aa-4f4d-96c7-6bae4260bb2d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:08.562856 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01a8e2-58aa-4f4d-96c7-6bae4260bb2d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:08.567178 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129034ec-4a89-4111-b0c4-eb97192e1f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:08.567759 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129034ec-4a89-4111-b0c4-eb97192e1f25 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:08.568049 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129034ec-4a89-4111-b0c4-eb97192e1f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:08.568318 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129034ec-4a89-4111-b0c4-eb97192e1f25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:08.568958 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129034ec-4a89-4111-b0c4-eb97192e1f25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:08.570592 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129034ec-4a89-4111-b0c4-eb97192e1f25 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:08.581155 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 937/1849] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:19:08 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:19:08.581860 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:08.581983 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129034ec-4a89-4111-b0c4-eb97192e1f25 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:08.595208 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129034ec-4a89-4111-b0c4-eb97192e1f25 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:08.599881 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee7e8c-6df0-4692-bc8e-50b564732c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:08.600512 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee7e8c-6df0-4692-bc8e-50b564732c0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:08.600796 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee7e8c-6df0-4692-bc8e-50b564732c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:08.601117 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee7e8c-6df0-4692-bc8e-50b564732c0e admin admin] PATH_INFO: `/v3/projects/585106b8643a492287683e4512066a7a/users/be7f18241ce64008a871f628bd0810fe/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:08.606560 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3424482f-f6fe-4eff-8096-615540b3f1ec admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:08.611514 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e2a855-9dde-43c0-a8e1-e55e7d77c032 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:08.612177 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2a855-9dde-43c0-a8e1-e55e7d77c032 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:08.612475 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2a855-9dde-43c0-a8e1-e55e7d77c032 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:08.612742 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2a855-9dde-43c0-a8e1-e55e7d77c032 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:08.615743 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-09e2a855-9dde-43c0-a8e1-e55e7d77c032 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:08.619155 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ee7e8c-6df0-4692-bc8e-50b564732c0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=585106b8643a492287683e4512066a7a, user_id=be7f18241ce64008a871f628bd0810fe, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:08.621845 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ee7e8c-6df0-4692-bc8e-50b564732c0e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:08.645183 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 938/1850] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:19:08 2022] PUT /identity/v3/projects/585106b8643a492287683e4512066a7a/users/be7f18241ce64008a871f628bd0810fe/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:19:08.646416 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee7e8c-6df0-4692-bc8e-50b564732c0e admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:08.693030 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e2a855-9dde-43c0-a8e1-e55e7d77c032 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:08.694681 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e2a855-9dde-43c0-a8e1-e55e7d77c032 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:08.704843 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 913/1851] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:19:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:19:08.706487 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82154a-8e2b-4bd1-9782-5e6d0e236445 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:08.706881 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82154a-8e2b-4bd1-9782-5e6d0e236445 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:08.707224 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82154a-8e2b-4bd1-9782-5e6d0e236445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:08.735318 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-3f82154a-8e2b-4bd1-9782-5e6d0e236445 None None] MFA Rules not processed for user `be7f18241ce64008a871f628bd0810fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:19:08.738475 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3f82154a-8e2b-4bd1-9782-5e6d0e236445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:08.743498 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee7e8c-6df0-4692-bc8e-50b564732c0e admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:08.752489 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb5ea10-e07d-4ad7-8267-2eb10d4d8f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:08.752489 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5ea10-e07d-4ad7-8267-2eb10d4d8f93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:08.752812 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5ea10-e07d-4ad7-8267-2eb10d4d8f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:08.753091 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5ea10-e07d-4ad7-8267-2eb10d4d8f93 admin admin] PATH_INFO: `/v3/projects/0d8cc9a55beb49098a0a489fd69450f1/users/8852f6fab4ba467d94cdec7f358b192f/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:08.756590 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:08.785089 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb5ea10-e07d-4ad7-8267-2eb10d4d8f93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d8cc9a55beb49098a0a489fd69450f1, user_id=8852f6fab4ba467d94cdec7f358b192f, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:08.788238 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb5ea10-e07d-4ad7-8267-2eb10d4d8f93 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:08.806163 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 939/1852] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:19:08 2022] PUT /identity/v3/projects/0d8cc9a55beb49098a0a489fd69450f1/users/8852f6fab4ba467d94cdec7f358b192f/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:19:08.844141 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fa93e-6229-4c7b-b803-9a2d39b867b4 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:08.845463 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 914/1853] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:19:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:19:08.846157 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f82154a-8e2b-4bd1-9782-5e6d0e236445 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:08.847064 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fa93e-6229-4c7b-b803-9a2d39b867b4 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:08.847388 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fa93e-6229-4c7b-b803-9a2d39b867b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:08.864451 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f82154a-8e2b-4bd1-9782-5e6d0e236445 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:08.902742 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-419fa93e-6229-4c7b-b803-9a2d39b867b4 None None] MFA Rules not processed for user `8852f6fab4ba467d94cdec7f358b192f`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:19:08.903974 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-419fa93e-6229-4c7b-b803-9a2d39b867b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:09.004229 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc02bf-f613-4483-be4d-cf761a63671b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:09.005238 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc02bf-f613-4483-be4d-cf761a63671b admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:09.005527 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc02bf-f613-4483-be4d-cf761a63671b admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:09.005798 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc02bf-f613-4483-be4d-cf761a63671b admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:09.006496 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cc02bf-f613-4483-be4d-cf761a63671b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:09.009326 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cc02bf-f613-4483-be4d-cf761a63671b admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:09.034642 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 940/1854] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:19:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:19:09.041964 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419fa93e-6229-4c7b-b803-9a2d39b867b4 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:09.043543 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-419fa93e-6229-4c7b-b803-9a2d39b867b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:09.047638 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 915/1855] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:19:08 2022] POST /identity/v3/projects => generated 379 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:19:09.048464 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc02bf-f613-4483-be4d-cf761a63671b admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:09.064315 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:09.129297 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc02bf-f613-4483-be4d-cf761a63671b admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:09.134917 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a9782-b808-45f8-a934-98f9dced2760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:09.135568 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a9782-b808-45f8-a934-98f9dced2760 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:09.136148 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a9782-b808-45f8-a934-98f9dced2760 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:09.136488 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a9782-b808-45f8-a934-98f9dced2760 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:09.137398 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5a9782-b808-45f8-a934-98f9dced2760 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:09.139892 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5a9782-b808-45f8-a934-98f9dced2760 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:09.142612 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419fa93e-6229-4c7b-b803-9a2d39b867b4 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:09.166470 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 916/1856] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:19:09 2022] POST /identity/v3/users => generated 296 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:19:09.169050 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a9782-b808-45f8-a934-98f9dced2760 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:09.179432 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a9782-b808-45f8-a934-98f9dced2760 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:09.184423 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5656f4e6-6c3b-4b73-b8aa-a525c2814ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:09.185074 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5656f4e6-6c3b-4b73-b8aa-a525c2814ab7 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:09.185358 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5656f4e6-6c3b-4b73-b8aa-a525c2814ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:09.185625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5656f4e6-6c3b-4b73-b8aa-a525c2814ab7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:09.186378 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5656f4e6-6c3b-4b73-b8aa-a525c2814ab7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:09.188374 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5656f4e6-6c3b-4b73-b8aa-a525c2814ab7 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:09.199739 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 917/1857] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 05:19:09.203883 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5656f4e6-6c3b-4b73-b8aa-a525c2814ab7 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:09.210253 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde6c826-9266-41bf-aa1f-c7eda78a5de5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:09.210871 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde6c826-9266-41bf-aa1f-c7eda78a5de5 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:09.211527 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde6c826-9266-41bf-aa1f-c7eda78a5de5 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:09.211806 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde6c826-9266-41bf-aa1f-c7eda78a5de5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:09.213322 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5656f4e6-6c3b-4b73-b8aa-a525c2814ab7 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:09.217257 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:09.217766 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730b299-5996-4843-8d97-a98cc27b2755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:09.218498 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730b299-5996-4843-8d97-a98cc27b2755 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:09.218818 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730b299-5996-4843-8d97-a98cc27b2755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:09.219133 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730b299-5996-4843-8d97-a98cc27b2755 admin admin] PATH_INFO: `/v3/projects/9d26959b2c874c3a8a46e7dbbbd1b2e3/users/d9dfdb1d963e493bbcf28cea3fd90b67/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:09.233527 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730b299-5996-4843-8d97-a98cc27b2755 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d26959b2c874c3a8a46e7dbbbd1b2e3, user_id=d9dfdb1d963e493bbcf28cea3fd90b67, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:09.236605 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730b299-5996-4843-8d97-a98cc27b2755 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:09.251226 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 918/1858] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:19:09 2022] PUT /identity/v3/projects/9d26959b2c874c3a8a46e7dbbbd1b2e3/users/d9dfdb1d963e493bbcf28cea3fd90b67/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:19:09.255013 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730b299-5996-4843-8d97-a98cc27b2755 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:09.269860 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde6c826-9266-41bf-aa1f-c7eda78a5de5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:09.271503 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde6c826-9266-41bf-aa1f-c7eda78a5de5 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:09.280436 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 941/1859] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:19:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:19:09.321401 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730b299-5996-4843-8d97-a98cc27b2755 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:09.326674 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e94b11-2dfe-4363-85a6-9cd8d71a2410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:09.327284 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e94b11-2dfe-4363-85a6-9cd8d71a2410 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:09.327539 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e94b11-2dfe-4363-85a6-9cd8d71a2410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:09.327832 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e94b11-2dfe-4363-85a6-9cd8d71a2410 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:09.328476 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e94b11-2dfe-4363-85a6-9cd8d71a2410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:09.330116 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e94b11-2dfe-4363-85a6-9cd8d71a2410 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:09.343184 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 919/1860] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 05:19:09.347501 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde6c826-9266-41bf-aa1f-c7eda78a5de5 service nova] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:09.358515 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde6c826-9266-41bf-aa1f-c7eda78a5de5 service nova] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:09.363847 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd2826-6190-4fb1-bffe-adaddea05409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:09.364510 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd2826-6190-4fb1-bffe-adaddea05409 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:09.364786 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd2826-6190-4fb1-bffe-adaddea05409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:09.365054 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd2826-6190-4fb1-bffe-adaddea05409 admin admin] PATH_INFO: `/v3/projects/9d26959b2c874c3a8a46e7dbbbd1b2e3/users/d9dfdb1d963e493bbcf28cea3fd90b67/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:09.366981 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bd2826-6190-4fb1-bffe-adaddea05409 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d26959b2c874c3a8a46e7dbbbd1b2e3, user_id=d9dfdb1d963e493bbcf28cea3fd90b67, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:09.369806 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bd2826-6190-4fb1-bffe-adaddea05409 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:09.394613 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 942/1861] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:19:09 2022] PUT /identity/v3/projects/9d26959b2c874c3a8a46e7dbbbd1b2e3/users/d9dfdb1d963e493bbcf28cea3fd90b67/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:19:09.406098 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb244021-99f7-4fe6-838d-35491aa87533 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:09.406505 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb244021-99f7-4fe6-838d-35491aa87533 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:09.406834 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb244021-99f7-4fe6-838d-35491aa87533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:09.467060 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-eb244021-99f7-4fe6-838d-35491aa87533 None None] MFA Rules not processed for user `d9dfdb1d963e493bbcf28cea3fd90b67`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:19:09.468324 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-eb244021-99f7-4fe6-838d-35491aa87533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:09.536202 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd2826-6190-4fb1-bffe-adaddea05409 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:09.544940 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:09.579910 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 920/1862] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:19:09 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:19:09.589538 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb244021-99f7-4fe6-838d-35491aa87533 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:09.591241 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-eb244021-99f7-4fe6-838d-35491aa87533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:09.603807 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd2826-6190-4fb1-bffe-adaddea05409 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:09.610647 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b3f50-ff2f-48c2-a257-972a932824ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:09.611625 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b3f50-ff2f-48c2-a257-972a932824ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:09.611900 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b3f50-ff2f-48c2-a257-972a932824ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:09.612187 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b3f50-ff2f-48c2-a257-972a932824ce admin admin] PATH_INFO: `/v3/users/8852f6fab4ba467d94cdec7f358b192f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:09.613291 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5b3f50-ff2f-48c2-a257-972a932824ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=8852f6fab4ba467d94cdec7f358b192f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:09.615303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5b3f50-ff2f-48c2-a257-972a932824ce admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:09.667631 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4d5b3f50-ff2f-48c2-a257-972a932824ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8852f6fab4ba467d94cdec7f358b192f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:09.675304 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4d5b3f50-ff2f-48c2-a257-972a932824ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8852f6fab4ba467d94cdec7f358b192f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:09.676535 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb244021-99f7-4fe6-838d-35491aa87533 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:09.682202 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5e900b-0ab3-48c5-b7d1-0831887cd010 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:09.682960 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e900b-0ab3-48c5-b7d1-0831887cd010 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:09.683299 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e900b-0ab3-48c5-b7d1-0831887cd010 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:09.683641 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e900b-0ab3-48c5-b7d1-0831887cd010 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:09.684448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4d5b3f50-ff2f-48c2-a257-972a932824ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8852f6fab4ba467d94cdec7f358b192f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:09.685157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4d5b3f50-ff2f-48c2-a257-972a932824ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8852f6fab4ba467d94cdec7f358b192f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:09.693636 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5e900b-0ab3-48c5-b7d1-0831887cd010 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:09.695283 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5e900b-0ab3-48c5-b7d1-0831887cd010 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:09.701589 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4d5b3f50-ff2f-48c2-a257-972a932824ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8852f6fab4ba467d94cdec7f358b192f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:09.708656 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 921/1863] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:19:09 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 05:19:09.712850 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 943/1864] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:19:09 2022] DELETE /identity/v3/users/8852f6fab4ba467d94cdec7f358b192f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:19:09.714104 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd03da-cf3a-49a2-8bd1-a70e877fbc87 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:09.714588 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd03da-cf3a-49a2-8bd1-a70e877fbc87 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:09.714864 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd03da-cf3a-49a2-8bd1-a70e877fbc87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:09.742835 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-5fcd03da-cf3a-49a2-8bd1-a70e877fbc87 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:19:09.744388 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-5fcd03da-cf3a-49a2-8bd1-a70e877fbc87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:09.814070 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 922/1865] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:19:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:19:09.824632 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b3f50-ff2f-48c2-a257-972a932824ce admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:09.826168 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4d5b3f50-ff2f-48c2-a257-972a932824ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:09.841502 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:09.887819 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b3f50-ff2f-48c2-a257-972a932824ce admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:09.891736 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f97ce4-774c-447a-b211-c8b63cf0743c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:09.892340 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f97ce4-774c-447a-b211-c8b63cf0743c service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:09.892630 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f97ce4-774c-447a-b211-c8b63cf0743c service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:09.892914 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f97ce4-774c-447a-b211-c8b63cf0743c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:09.901770 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f97ce4-774c-447a-b211-c8b63cf0743c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:09.903238 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f97ce4-774c-447a-b211-c8b63cf0743c service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:09.912614 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 944/1866] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:19:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:19:10.098800 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcd03da-cf3a-49a2-8bd1-a70e877fbc87 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:10.100521 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-5fcd03da-cf3a-49a2-8bd1-a70e877fbc87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:10.219921 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f97ce4-774c-447a-b211-c8b63cf0743c service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:10.221678 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-01f97ce4-774c-447a-b211-c8b63cf0743c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:10.222783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcd03da-cf3a-49a2-8bd1-a70e877fbc87 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:10.227638 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d07479-00f4-446a-ba72-67d7b733037d tempest-VolumeQuotaClassesTest-154393190 tempest-VolumeQuotaClassesTest-154393190-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9dfdb1d963e493bbcf28cea3fd90b67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9d26959b2c874c3a8a46e7dbbbd1b2e3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:10.228339 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d07479-00f4-446a-ba72-67d7b733037d tempest-VolumeQuotaClassesTest-154393190 tempest-VolumeQuotaClassesTest-154393190-project-admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:10.228648 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d07479-00f4-446a-ba72-67d7b733037d tempest-VolumeQuotaClassesTest-154393190 tempest-VolumeQuotaClassesTest-154393190-project-admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:10.228920 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d07479-00f4-446a-ba72-67d7b733037d tempest-VolumeQuotaClassesTest-154393190 tempest-VolumeQuotaClassesTest-154393190-project-admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:10.229641 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d07479-00f4-446a-ba72-67d7b733037d tempest-VolumeQuotaClassesTest-154393190 tempest-VolumeQuotaClassesTest-154393190-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:19:10.229947 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d07479-00f4-446a-ba72-67d7b733037d tempest-VolumeQuotaClassesTest-154393190 tempest-VolumeQuotaClassesTest-154393190-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:10.231576 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d07479-00f4-446a-ba72-67d7b733037d tempest-VolumeQuotaClassesTest-154393190 tempest-VolumeQuotaClassesTest-154393190-project-admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:10.241319 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f97ce4-774c-447a-b211-c8b63cf0743c service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:10.251516 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 923/1867] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:19:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:19:10.252937 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d07479-00f4-446a-ba72-67d7b733037d tempest-VolumeQuotaClassesTest-154393190 tempest-VolumeQuotaClassesTest-154393190-project-admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:10.276063 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d60d7-da7e-424a-b1b3-39261fe7cc28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:10.276600 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d60d7-da7e-424a-b1b3-39261fe7cc28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:10.277030 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d60d7-da7e-424a-b1b3-39261fe7cc28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:10.277318 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d60d7-da7e-424a-b1b3-39261fe7cc28 admin admin] PATH_INFO: `/v3/projects/0d8cc9a55beb49098a0a489fd69450f1` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:10.279467 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4d60d7-da7e-424a-b1b3-39261fe7cc28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d8cc9a55beb49098a0a489fd69450f1)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:10.282171 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4d60d7-da7e-424a-b1b3-39261fe7cc28 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:10.284076 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d07479-00f4-446a-ba72-67d7b733037d tempest-VolumeQuotaClassesTest-154393190 tempest-VolumeQuotaClassesTest-154393190-project-admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:10.292646 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d8bcc-0dd3-4245-8422-23b39b02521c tempest-VolumeQuotaClassesTest-154393190 tempest-VolumeQuotaClassesTest-154393190-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9dfdb1d963e493bbcf28cea3fd90b67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9d26959b2c874c3a8a46e7dbbbd1b2e3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:10.293368 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d8bcc-0dd3-4245-8422-23b39b02521c tempest-VolumeQuotaClassesTest-154393190 tempest-VolumeQuotaClassesTest-154393190-project-admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:10.293618 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d8bcc-0dd3-4245-8422-23b39b02521c tempest-VolumeQuotaClassesTest-154393190 tempest-VolumeQuotaClassesTest-154393190-project-admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:10.293878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d8bcc-0dd3-4245-8422-23b39b02521c tempest-VolumeQuotaClassesTest-154393190 tempest-VolumeQuotaClassesTest-154393190-project-admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:10.294769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4d8bcc-0dd3-4245-8422-23b39b02521c tempest-VolumeQuotaClassesTest-154393190 tempest-VolumeQuotaClassesTest-154393190-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:10.296888 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4d8bcc-0dd3-4245-8422-23b39b02521c tempest-VolumeQuotaClassesTest-154393190 tempest-VolumeQuotaClassesTest-154393190-project-admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:10.321162 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:10.332061 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 924/1868] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:19:10 2022] POST /identity/v3/projects => generated 353 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:19:10.333533 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d8bcc-0dd3-4245-8422-23b39b02521c tempest-VolumeQuotaClassesTest-154393190 tempest-VolumeQuotaClassesTest-154393190-project-admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:10.346038 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-da4d60d7-da7e-424a-b1b3-39261fe7cc28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d8cc9a55beb49098a0a489fd69450f1', 'request_id': 'req-da4d60d7-da7e-424a-b1b3-39261fe7cc28'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:10.354137 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-da4d60d7-da7e-424a-b1b3-39261fe7cc28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d8cc9a55beb49098a0a489fd69450f1', 'request_id': 'req-da4d60d7-da7e-424a-b1b3-39261fe7cc28'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:10.366323 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-da4d60d7-da7e-424a-b1b3-39261fe7cc28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d8cc9a55beb49098a0a489fd69450f1', 'request_id': 'req-da4d60d7-da7e-424a-b1b3-39261fe7cc28'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:10.383010 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-da4d60d7-da7e-424a-b1b3-39261fe7cc28 admin admin] The token cache is being invalidate because project 0d8cc9a55beb49098a0a489fd69450f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:19:10.383552 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-da4d60d7-da7e-424a-b1b3-39261fe7cc28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:10.384510 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-da4d60d7-da7e-424a-b1b3-39261fe7cc28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:10.386467 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 945/1869] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:19:10 2022] DELETE /identity/v3/projects/0d8cc9a55beb49098a0a489fd69450f1 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:19:10.392268 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757dd988-cf6a-4baf-acbf-a06a3edc4951 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:10.392591 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757dd988-cf6a-4baf-acbf-a06a3edc4951 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:10.392886 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757dd988-cf6a-4baf-acbf-a06a3edc4951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:10.413018 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d8bcc-0dd3-4245-8422-23b39b02521c tempest-VolumeQuotaClassesTest-154393190 tempest-VolumeQuotaClassesTest-154393190-project-admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:10.413149 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-757dd988-cf6a-4baf-acbf-a06a3edc4951 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:19:10.414371 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-757dd988-cf6a-4baf-acbf-a06a3edc4951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:10.418395 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89fcb38-83ef-4e0d-9ae7-d46d313c4863 tempest-VolumeQuotaClassesTest-154393190 tempest-VolumeQuotaClassesTest-154393190-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9dfdb1d963e493bbcf28cea3fd90b67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9d26959b2c874c3a8a46e7dbbbd1b2e3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:10.419176 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89fcb38-83ef-4e0d-9ae7-d46d313c4863 tempest-VolumeQuotaClassesTest-154393190 tempest-VolumeQuotaClassesTest-154393190-project-admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:10.419454 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89fcb38-83ef-4e0d-9ae7-d46d313c4863 tempest-VolumeQuotaClassesTest-154393190 tempest-VolumeQuotaClassesTest-154393190-project-admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:10.422674 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89fcb38-83ef-4e0d-9ae7-d46d313c4863 tempest-VolumeQuotaClassesTest-154393190 tempest-VolumeQuotaClassesTest-154393190-project-admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:10.422674 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89fcb38-83ef-4e0d-9ae7-d46d313c4863 tempest-VolumeQuotaClassesTest-154393190 tempest-VolumeQuotaClassesTest-154393190-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:19:10.422674 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89fcb38-83ef-4e0d-9ae7-d46d313c4863 tempest-VolumeQuotaClassesTest-154393190 tempest-VolumeQuotaClassesTest-154393190-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:10.424362 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89fcb38-83ef-4e0d-9ae7-d46d313c4863 tempest-VolumeQuotaClassesTest-154393190 tempest-VolumeQuotaClassesTest-154393190-project-admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:10.438038 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 925/1870] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:19:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:19:10.477833 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 946/1871] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:19:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:19:10.477998 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89fcb38-83ef-4e0d-9ae7-d46d313c4863 tempest-VolumeQuotaClassesTest-154393190 tempest-VolumeQuotaClassesTest-154393190-project-admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:10.479696 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c89fcb38-83ef-4e0d-9ae7-d46d313c4863 tempest-VolumeQuotaClassesTest-154393190 tempest-VolumeQuotaClassesTest-154393190-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:10.485187 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b299c1-c2f0-4cff-87e0-863bc1770a2e None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:10.485469 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b299c1-c2f0-4cff-87e0-863bc1770a2e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:10.485740 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b299c1-c2f0-4cff-87e0-863bc1770a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:10.493719 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89fcb38-83ef-4e0d-9ae7-d46d313c4863 tempest-VolumeQuotaClassesTest-154393190 tempest-VolumeQuotaClassesTest-154393190-project-admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:10.498661 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b43339-53dc-4d0f-9c43-488a813c835a tempest-VolumeQuotaClassesTest-154393190 tempest-VolumeQuotaClassesTest-154393190-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9dfdb1d963e493bbcf28cea3fd90b67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9d26959b2c874c3a8a46e7dbbbd1b2e3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:10.499284 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b43339-53dc-4d0f-9c43-488a813c835a tempest-VolumeQuotaClassesTest-154393190 tempest-VolumeQuotaClassesTest-154393190-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:10.500082 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b43339-53dc-4d0f-9c43-488a813c835a tempest-VolumeQuotaClassesTest-154393190 tempest-VolumeQuotaClassesTest-154393190-project-admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:10.500357 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b43339-53dc-4d0f-9c43-488a813c835a tempest-VolumeQuotaClassesTest-154393190 tempest-VolumeQuotaClassesTest-154393190-project-admin] PATH_INFO: `/v3/projects/e77850eab51d475a9248faea2405602e` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:10.501479 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b43339-53dc-4d0f-9c43-488a813c835a tempest-VolumeQuotaClassesTest-154393190 tempest-VolumeQuotaClassesTest-154393190-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e77850eab51d475a9248faea2405602e)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:10.503483 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b43339-53dc-4d0f-9c43-488a813c835a tempest-VolumeQuotaClassesTest-154393190 tempest-VolumeQuotaClassesTest-154393190-project-admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:10.505573 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-43b299c1-c2f0-4cff-87e0-863bc1770a2e None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:19:10.506796 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-43b299c1-c2f0-4cff-87e0-863bc1770a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:10.514899 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 947/1872] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:19:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:19:10.519057 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b299c1-c2f0-4cff-87e0-863bc1770a2e None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:10.528185 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b299c1-c2f0-4cff-87e0-863bc1770a2e None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:10.534171 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a051f-3fb0-4c6e-b802-3d9c34537dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:10.534868 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a051f-3fb0-4c6e-b802-3d9c34537dba admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:10.535649 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a051f-3fb0-4c6e-b802-3d9c34537dba admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:10.536089 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a051f-3fb0-4c6e-b802-3d9c34537dba admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:10.536885 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272a051f-3fb0-4c6e-b802-3d9c34537dba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:19:10.537427 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272a051f-3fb0-4c6e-b802-3d9c34537dba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:10.539281 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272a051f-3fb0-4c6e-b802-3d9c34537dba admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:10.551387 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 948/1873] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:19:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:19:10.553689 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-66b43339-53dc-4d0f-9c43-488a813c835a tempest-VolumeQuotaClassesTest-154393190 tempest-VolumeQuotaClassesTest-154393190-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e77850eab51d475a9248faea2405602e', 'request_id': 'req-66b43339-53dc-4d0f-9c43-488a813c835a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:10.555591 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a051f-3fb0-4c6e-b802-3d9c34537dba admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:10.560511 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-66b43339-53dc-4d0f-9c43-488a813c835a tempest-VolumeQuotaClassesTest-154393190 tempest-VolumeQuotaClassesTest-154393190-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e77850eab51d475a9248faea2405602e', 'request_id': 'req-66b43339-53dc-4d0f-9c43-488a813c835a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:10.565095 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a051f-3fb0-4c6e-b802-3d9c34537dba admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:10.566089 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-66b43339-53dc-4d0f-9c43-488a813c835a tempest-VolumeQuotaClassesTest-154393190 tempest-VolumeQuotaClassesTest-154393190-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e77850eab51d475a9248faea2405602e', 'request_id': 'req-66b43339-53dc-4d0f-9c43-488a813c835a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:10.582676 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:10.583474 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-66b43339-53dc-4d0f-9c43-488a813c835a tempest-VolumeQuotaClassesTest-154393190 tempest-VolumeQuotaClassesTest-154393190-project-admin] The token cache is being invalidate because project e77850eab51d475a9248faea2405602e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:19:10.583736 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-66b43339-53dc-4d0f-9c43-488a813c835a tempest-VolumeQuotaClassesTest-154393190 tempest-VolumeQuotaClassesTest-154393190-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:10.584469 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-66b43339-53dc-4d0f-9c43-488a813c835a tempest-VolumeQuotaClassesTest-154393190 tempest-VolumeQuotaClassesTest-154393190-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:10.589609 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 926/1874] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:19:10 2022] DELETE /identity/v3/projects/e77850eab51d475a9248faea2405602e => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:19:10.627364 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60012732-4772-4133-b359-df906d2fb2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:10.628058 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60012732-4772-4133-b359-df906d2fb2c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:10.628248 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60012732-4772-4133-b359-df906d2fb2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:10.628507 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60012732-4772-4133-b359-df906d2fb2c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:10.629147 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60012732-4772-4133-b359-df906d2fb2c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:10.631069 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60012732-4772-4133-b359-df906d2fb2c0 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:10.662768 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 949/1875] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:19:10 2022] POST /identity/v3/projects => generated 355 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:19:10.667460 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b43339-53dc-4d0f-9c43-488a813c835a tempest-VolumeQuotaClassesTest-154393190 tempest-VolumeQuotaClassesTest-154393190-project-admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:10.672030 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-66b43339-53dc-4d0f-9c43-488a813c835a tempest-VolumeQuotaClassesTest-154393190 tempest-VolumeQuotaClassesTest-154393190-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:10.702297 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b43339-53dc-4d0f-9c43-488a813c835a tempest-VolumeQuotaClassesTest-154393190 tempest-VolumeQuotaClassesTest-154393190-project-admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:10.765141 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9b58b-4bdb-4f2a-8d1a-fe219d268922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:10.765944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9b58b-4bdb-4f2a-8d1a-fe219d268922 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:10.766249 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9b58b-4bdb-4f2a-8d1a-fe219d268922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:10.766517 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9b58b-4bdb-4f2a-8d1a-fe219d268922 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:10.767157 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b9b58b-4bdb-4f2a-8d1a-fe219d268922 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:10.769419 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b9b58b-4bdb-4f2a-8d1a-fe219d268922 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:10.804917 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 927/1876] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:19:10 2022] POST /identity/v3/users => generated 278 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:19:10.805949 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60012732-4772-4133-b359-df906d2fb2c0 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:10.817342 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60012732-4772-4133-b359-df906d2fb2c0 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:10.823977 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da5724-0678-45f6-8676-414a1064293e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:10.824594 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da5724-0678-45f6-8676-414a1064293e admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:10.824867 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da5724-0678-45f6-8676-414a1064293e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:10.825263 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da5724-0678-45f6-8676-414a1064293e admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:10.825918 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da5724-0678-45f6-8676-414a1064293e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:10.827744 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da5724-0678-45f6-8676-414a1064293e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:10.849361 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 950/1877] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:19:10 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 05:19:10.853454 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9b58b-4bdb-4f2a-8d1a-fe219d268922 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:10.865020 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9b58b-4bdb-4f2a-8d1a-fe219d268922 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:10.870151 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da5724-0678-45f6-8676-414a1064293e admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:10.870601 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d885e83d-aeae-45a1-b1f2-5b66d6561d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:10.873678 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d885e83d-aeae-45a1-b1f2-5b66d6561d41 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:10.873678 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d885e83d-aeae-45a1-b1f2-5b66d6561d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:10.873678 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d885e83d-aeae-45a1-b1f2-5b66d6561d41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:10.873678 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d885e83d-aeae-45a1-b1f2-5b66d6561d41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:10.874092 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d885e83d-aeae-45a1-b1f2-5b66d6561d41 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:10.874470 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-96da5724-0678-45f6-8676-414a1064293e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:10.875540 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:10.888903 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 928/1878] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:19:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:19:10.889890 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d885e83d-aeae-45a1-b1f2-5b66d6561d41 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:10.891401 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da5724-0678-45f6-8676-414a1064293e admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:10.896253 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e6d2a-9208-484c-ba76-abc71b13e588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:10.897427 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e6d2a-9208-484c-ba76-abc71b13e588 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:10.897711 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e6d2a-9208-484c-ba76-abc71b13e588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:10.897989 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e6d2a-9208-484c-ba76-abc71b13e588 admin admin] PATH_INFO: `/v3/users/be7f18241ce64008a871f628bd0810fe` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:10.899213 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622e6d2a-9208-484c-ba76-abc71b13e588 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be7f18241ce64008a871f628bd0810fe)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:10.901183 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622e6d2a-9208-484c-ba76-abc71b13e588 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:10.902093 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d885e83d-aeae-45a1-b1f2-5b66d6561d41 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:10.912776 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ecdff6-078e-4ab2-ad6c-c78d539c182b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:10.913676 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ecdff6-078e-4ab2-ad6c-c78d539c182b admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:10.914047 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ecdff6-078e-4ab2-ad6c-c78d539c182b admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:10.914472 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ecdff6-078e-4ab2-ad6c-c78d539c182b admin admin] PATH_INFO: `/v3/projects/2252d9e0337446b88dc9d3c929683b62/users/57fd6909d759418ab043936625d0680e/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:10.930860 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ecdff6-078e-4ab2-ad6c-c78d539c182b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2252d9e0337446b88dc9d3c929683b62, user_id=57fd6909d759418ab043936625d0680e, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:10.934479 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ecdff6-078e-4ab2-ad6c-c78d539c182b admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:10.955160 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 929/1879] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:19:10 2022] PUT /identity/v3/projects/2252d9e0337446b88dc9d3c929683b62/users/57fd6909d759418ab043936625d0680e/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:19:10.964358 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-622e6d2a-9208-484c-ba76-abc71b13e588 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be7f18241ce64008a871f628bd0810fe'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:10.965594 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d57ec0-a83c-4a83-967c-c8634ce8a013 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:10.965882 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d57ec0-a83c-4a83-967c-c8634ce8a013 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:10.966269 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d57ec0-a83c-4a83-967c-c8634ce8a013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:10.981277 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-622e6d2a-9208-484c-ba76-abc71b13e588 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be7f18241ce64008a871f628bd0810fe'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:10.996747 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-66d57ec0-a83c-4a83-967c-c8634ce8a013 None None] MFA Rules not processed for user `57fd6909d759418ab043936625d0680e`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:19:10.998189 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-66d57ec0-a83c-4a83-967c-c8634ce8a013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:11.000774 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-622e6d2a-9208-484c-ba76-abc71b13e588 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be7f18241ce64008a871f628bd0810fe'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:11.001414 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-622e6d2a-9208-484c-ba76-abc71b13e588 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be7f18241ce64008a871f628bd0810fe'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:11.014245 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-622e6d2a-9208-484c-ba76-abc71b13e588 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be7f18241ce64008a871f628bd0810fe'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:11.020341 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 951/1880] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:19:10 2022] DELETE /identity/v3/users/be7f18241ce64008a871f628bd0810fe => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:19:11.024749 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df95e0-3481-442c-874b-5aed933cce6b None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:11.025163 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df95e0-3481-442c-874b-5aed933cce6b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:11.025597 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df95e0-3481-442c-874b-5aed933cce6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:11.046023 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-41df95e0-3481-442c-874b-5aed933cce6b None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:19:11.047399 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-41df95e0-3481-442c-874b-5aed933cce6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:11.065444 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 930/1881] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:19:10 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:19:11.069904 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d57ec0-a83c-4a83-967c-c8634ce8a013 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:11.071262 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-66d57ec0-a83c-4a83-967c-c8634ce8a013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:11.106341 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 952/1882] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:19:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:19:11.122856 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df95e0-3481-442c-874b-5aed933cce6b None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:11.124922 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-41df95e0-3481-442c-874b-5aed933cce6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:11.138502 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:11.152473 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d57ec0-a83c-4a83-967c-c8634ce8a013 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:11.159039 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97901053-9422-4bd3-b5df-85ee0428caa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:11.159672 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97901053-9422-4bd3-b5df-85ee0428caa0 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:11.159945 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97901053-9422-4bd3-b5df-85ee0428caa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:11.160291 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97901053-9422-4bd3-b5df-85ee0428caa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:11.161026 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97901053-9422-4bd3-b5df-85ee0428caa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:11.163097 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97901053-9422-4bd3-b5df-85ee0428caa0 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:11.194359 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 931/1883] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:19:11 2022] POST /identity/v3/projects => generated 357 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:19:11.199760 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97901053-9422-4bd3-b5df-85ee0428caa0 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:11.217798 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df95e0-3481-442c-874b-5aed933cce6b None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:11.267964 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:11.287155 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97901053-9422-4bd3-b5df-85ee0428caa0 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:11.292669 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61c60d3-ff2b-4b0b-ae3f-f168fb66e3b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:11.293334 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c60d3-ff2b-4b0b-ae3f-f168fb66e3b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:11.293610 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c60d3-ff2b-4b0b-ae3f-f168fb66e3b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:11.293878 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c60d3-ff2b-4b0b-ae3f-f168fb66e3b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:11.293976 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc8889-607a-4908-8fd6-c6ea1fd8892d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:11.295756 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc8889-607a-4908-8fd6-c6ea1fd8892d admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:11.295756 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc8889-607a-4908-8fd6-c6ea1fd8892d admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:11.295756 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc8889-607a-4908-8fd6-c6ea1fd8892d admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:11.295756 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acc8889-607a-4908-8fd6-c6ea1fd8892d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:11.297855 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acc8889-607a-4908-8fd6-c6ea1fd8892d admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:11.304680 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61c60d3-ff2b-4b0b-ae3f-f168fb66e3b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:11.306398 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61c60d3-ff2b-4b0b-ae3f-f168fb66e3b6 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:11.317556 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 953/1884] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:19:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:19:11.322723 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 932/1885] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:19:11 2022] POST /identity/v3/users => generated 285 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:19:11.327303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61c60d3-ff2b-4b0b-ae3f-f168fb66e3b6 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:11.338091 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61c60d3-ff2b-4b0b-ae3f-f168fb66e3b6 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:11.345945 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c649234b-479a-40f2-8f96-581938a2c54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:11.347022 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649234b-479a-40f2-8f96-581938a2c54c admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:11.347612 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649234b-479a-40f2-8f96-581938a2c54c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:11.348084 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649234b-479a-40f2-8f96-581938a2c54c admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:11.349164 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c649234b-479a-40f2-8f96-581938a2c54c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:11.351222 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c649234b-479a-40f2-8f96-581938a2c54c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:11.363067 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 954/1886] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 05:19:11.367398 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc8889-607a-4908-8fd6-c6ea1fd8892d admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:11.378741 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc8889-607a-4908-8fd6-c6ea1fd8892d admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:11.384001 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f63517-7a1d-4071-a523-ed098e8c5bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:11.384699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f63517-7a1d-4071-a523-ed098e8c5bbc admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:11.384975 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f63517-7a1d-4071-a523-ed098e8c5bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:11.385244 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f63517-7a1d-4071-a523-ed098e8c5bbc admin admin] PATH_INFO: `/v3/projects/683a709a8e804ffe851db24f709b5b48/users/90732a9d0b034748879dbb1da4855d5b/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:11.401741 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f63517-7a1d-4071-a523-ed098e8c5bbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=683a709a8e804ffe851db24f709b5b48, user_id=90732a9d0b034748879dbb1da4855d5b, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:11.404483 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f63517-7a1d-4071-a523-ed098e8c5bbc admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:11.420801 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 933/1887] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:19:11 2022] PUT /identity/v3/projects/683a709a8e804ffe851db24f709b5b48/users/90732a9d0b034748879dbb1da4855d5b/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:19:11.425301 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c649234b-479a-40f2-8f96-581938a2c54c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:11.436328 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c649234b-479a-40f2-8f96-581938a2c54c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:11.502707 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e3a4d-f477-4d03-8d16-cb224ad02c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:11.503330 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e3a4d-f477-4d03-8d16-cb224ad02c60 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:11.503608 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e3a4d-f477-4d03-8d16-cb224ad02c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:11.503876 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e3a4d-f477-4d03-8d16-cb224ad02c60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:11.504553 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276e3a4d-f477-4d03-8d16-cb224ad02c60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:11.506199 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276e3a4d-f477-4d03-8d16-cb224ad02c60 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:11.519552 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 955/1888] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:19:11.520948 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f63517-7a1d-4071-a523-ed098e8c5bbc admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:11.531616 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f63517-7a1d-4071-a523-ed098e8c5bbc admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:11.536257 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eada067-1644-40b3-b74a-87b058595e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:11.536926 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eada067-1644-40b3-b74a-87b058595e64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:11.537187 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eada067-1644-40b3-b74a-87b058595e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:11.537454 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eada067-1644-40b3-b74a-87b058595e64 admin admin] PATH_INFO: `/v3/projects/683a709a8e804ffe851db24f709b5b48/users/90732a9d0b034748879dbb1da4855d5b/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:11.539395 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eada067-1644-40b3-b74a-87b058595e64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=683a709a8e804ffe851db24f709b5b48, user_id=90732a9d0b034748879dbb1da4855d5b, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:11.542082 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eada067-1644-40b3-b74a-87b058595e64 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:11.554509 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 934/1889] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:19:11 2022] PUT /identity/v3/projects/683a709a8e804ffe851db24f709b5b48/users/90732a9d0b034748879dbb1da4855d5b/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:19:11.566521 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca578275-574f-412c-8e6d-6107d8593c6e None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:11.566903 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca578275-574f-412c-8e6d-6107d8593c6e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:11.567244 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca578275-574f-412c-8e6d-6107d8593c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:11.600509 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ca578275-574f-412c-8e6d-6107d8593c6e None None] MFA Rules not processed for user `90732a9d0b034748879dbb1da4855d5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:19:11.601812 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ca578275-574f-412c-8e6d-6107d8593c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:11.609222 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eada067-1644-40b3-b74a-87b058595e64 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:11.611580 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-6eada067-1644-40b3-b74a-87b058595e64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:11.658283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:11.681500 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eada067-1644-40b3-b74a-87b058595e64 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:11.686404 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec4a320-ef12-4b5d-9173-7236cbc7aaf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:11.687060 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4a320-ef12-4b5d-9173-7236cbc7aaf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:11.687337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4a320-ef12-4b5d-9173-7236cbc7aaf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:11.687606 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4a320-ef12-4b5d-9173-7236cbc7aaf3 admin admin] PATH_INFO: `/v3/projects/585106b8643a492287683e4512066a7a` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:11.688621 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 956/1890] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:19:11 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:19:11.688774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec4a320-ef12-4b5d-9173-7236cbc7aaf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=585106b8643a492287683e4512066a7a)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:11.690851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec4a320-ef12-4b5d-9173-7236cbc7aaf3 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:11.699719 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca578275-574f-412c-8e6d-6107d8593c6e None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:11.701552 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ca578275-574f-412c-8e6d-6107d8593c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:11.739893 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-8ec4a320-ef12-4b5d-9173-7236cbc7aaf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '585106b8643a492287683e4512066a7a', 'request_id': 'req-8ec4a320-ef12-4b5d-9173-7236cbc7aaf3'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:11.746442 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-8ec4a320-ef12-4b5d-9173-7236cbc7aaf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '585106b8643a492287683e4512066a7a', 'request_id': 'req-8ec4a320-ef12-4b5d-9173-7236cbc7aaf3'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:11.751874 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-8ec4a320-ef12-4b5d-9173-7236cbc7aaf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '585106b8643a492287683e4512066a7a', 'request_id': 'req-8ec4a320-ef12-4b5d-9173-7236cbc7aaf3'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:11.758393 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca578275-574f-412c-8e6d-6107d8593c6e None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:11.766146 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-8ec4a320-ef12-4b5d-9173-7236cbc7aaf3 admin admin] The token cache is being invalidate because project 585106b8643a492287683e4512066a7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:19:11.766442 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-8ec4a320-ef12-4b5d-9173-7236cbc7aaf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:11.767065 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-8ec4a320-ef12-4b5d-9173-7236cbc7aaf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:11.768683 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 935/1891] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:19:11 2022] DELETE /identity/v3/projects/585106b8643a492287683e4512066a7a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:19:11.772711 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec4a320-ef12-4b5d-9173-7236cbc7aaf3 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:11.774121 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-8ec4a320-ef12-4b5d-9173-7236cbc7aaf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:11.805004 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:11.807495 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09fbfb6-960f-44d0-9960-3e5d755e1c8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:11.808113 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09fbfb6-960f-44d0-9960-3e5d755e1c8c service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:11.808390 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09fbfb6-960f-44d0-9960-3e5d755e1c8c service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:11.808653 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09fbfb6-960f-44d0-9960-3e5d755e1c8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:11.810400 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c09fbfb6-960f-44d0-9960-3e5d755e1c8c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:11.849843 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec4a320-ef12-4b5d-9173-7236cbc7aaf3 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:11.857345 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37608248-42d8-4c0b-ad1a-1f8b987b52e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:11.858313 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37608248-42d8-4c0b-ad1a-1f8b987b52e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:11.859013 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37608248-42d8-4c0b-ad1a-1f8b987b52e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:11.859288 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37608248-42d8-4c0b-ad1a-1f8b987b52e8 admin admin] PATH_INFO: `/v3/users/d9dfdb1d963e493bbcf28cea3fd90b67` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:11.862143 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37608248-42d8-4c0b-ad1a-1f8b987b52e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9dfdb1d963e493bbcf28cea3fd90b67)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:11.865169 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37608248-42d8-4c0b-ad1a-1f8b987b52e8 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:11.877972 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09fbfb6-960f-44d0-9960-3e5d755e1c8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:11.879512 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09fbfb6-960f-44d0-9960-3e5d755e1c8c service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:11.899124 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 957/1892] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:19:11 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 05:19:11.935523 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-37608248-42d8-4c0b-ad1a-1f8b987b52e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9dfdb1d963e493bbcf28cea3fd90b67'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:11.944337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-37608248-42d8-4c0b-ad1a-1f8b987b52e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9dfdb1d963e493bbcf28cea3fd90b67'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:11.945115 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-37608248-42d8-4c0b-ad1a-1f8b987b52e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9dfdb1d963e493bbcf28cea3fd90b67'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:11.954060 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-37608248-42d8-4c0b-ad1a-1f8b987b52e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9dfdb1d963e493bbcf28cea3fd90b67'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:11.954775 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-37608248-42d8-4c0b-ad1a-1f8b987b52e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9dfdb1d963e493bbcf28cea3fd90b67'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:11.972723 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09fbfb6-960f-44d0-9960-3e5d755e1c8c service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:11.974887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c09fbfb6-960f-44d0-9960-3e5d755e1c8c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:11.978877 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 936/1893] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:19:11 2022] DELETE /identity/v3/users/d9dfdb1d963e493bbcf28cea3fd90b67 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:19:11.988254 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09fbfb6-960f-44d0-9960-3e5d755e1c8c service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:12.051041 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:12.062720 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254d0af-dc9b-4034-832e-24f27265747a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:12.063252 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254d0af-dc9b-4034-832e-24f27265747a service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:12.063733 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254d0af-dc9b-4034-832e-24f27265747a service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:12.063973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254d0af-dc9b-4034-832e-24f27265747a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:12.066097 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b254d0af-dc9b-4034-832e-24f27265747a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:12.141304 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254d0af-dc9b-4034-832e-24f27265747a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:12.142797 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254d0af-dc9b-4034-832e-24f27265747a service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:12.156916 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 958/1894] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:19:11 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 05:19:12.319737 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37608248-42d8-4c0b-ad1a-1f8b987b52e8 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:12.321212 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-37608248-42d8-4c0b-ad1a-1f8b987b52e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:12.405672 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37608248-42d8-4c0b-ad1a-1f8b987b52e8 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:12.410387 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75696d-1254-49dc-b6ff-9252e23cbc17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:12.411026 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75696d-1254-49dc-b6ff-9252e23cbc17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:12.411300 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75696d-1254-49dc-b6ff-9252e23cbc17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:12.411570 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75696d-1254-49dc-b6ff-9252e23cbc17 admin admin] PATH_INFO: `/v3/projects/9d26959b2c874c3a8a46e7dbbbd1b2e3` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:12.412714 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce75696d-1254-49dc-b6ff-9252e23cbc17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d26959b2c874c3a8a46e7dbbbd1b2e3)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:12.414742 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce75696d-1254-49dc-b6ff-9252e23cbc17 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:12.466814 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ce75696d-1254-49dc-b6ff-9252e23cbc17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d26959b2c874c3a8a46e7dbbbd1b2e3', 'request_id': 'req-ce75696d-1254-49dc-b6ff-9252e23cbc17'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:12.468335 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:12.475026 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ce75696d-1254-49dc-b6ff-9252e23cbc17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d26959b2c874c3a8a46e7dbbbd1b2e3', 'request_id': 'req-ce75696d-1254-49dc-b6ff-9252e23cbc17'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:12.482905 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ce75696d-1254-49dc-b6ff-9252e23cbc17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d26959b2c874c3a8a46e7dbbbd1b2e3', 'request_id': 'req-ce75696d-1254-49dc-b6ff-9252e23cbc17'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:12.497753 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ce75696d-1254-49dc-b6ff-9252e23cbc17 admin admin] The token cache is being invalidate because project 9d26959b2c874c3a8a46e7dbbbd1b2e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:19:12.498092 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ce75696d-1254-49dc-b6ff-9252e23cbc17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:12.498712 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ce75696d-1254-49dc-b6ff-9252e23cbc17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:12.500243 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 937/1895] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:19:12 2022] DELETE /identity/v3/projects/9d26959b2c874c3a8a46e7dbbbd1b2e3 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:19:12.505851 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aecea6-1962-47b0-95bb-42b18f280477 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:12.506175 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aecea6-1962-47b0-95bb-42b18f280477 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:12.506450 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aecea6-1962-47b0-95bb-42b18f280477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:12.525881 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-41aecea6-1962-47b0-95bb-42b18f280477 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:19:12.527109 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-41aecea6-1962-47b0-95bb-42b18f280477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:12.584956 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 959/1896] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:19:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:19:12.597370 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3340e0c7-4c90-457f-860c-8f4b6f1ee6af None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:12.597721 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3340e0c7-4c90-457f-860c-8f4b6f1ee6af None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:12.598207 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3340e0c7-4c90-457f-860c-8f4b6f1ee6af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:12.622870 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-3340e0c7-4c90-457f-860c-8f4b6f1ee6af None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:19:12.624082 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3340e0c7-4c90-457f-860c-8f4b6f1ee6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:12.631954 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 938/1897] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:19:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:19:12.636325 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aecea6-1962-47b0-95bb-42b18f280477 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:12.646196 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aecea6-1962-47b0-95bb-42b18f280477 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:12.650972 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e3d144-dba6-4d9b-9574-a7fc43f5500a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:12.651583 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e3d144-dba6-4d9b-9574-a7fc43f5500a admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:12.651859 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e3d144-dba6-4d9b-9574-a7fc43f5500a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:12.652124 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e3d144-dba6-4d9b-9574-a7fc43f5500a admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:12.652799 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e3d144-dba6-4d9b-9574-a7fc43f5500a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:19:12.653098 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e3d144-dba6-4d9b-9574-a7fc43f5500a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:12.654759 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e3d144-dba6-4d9b-9574-a7fc43f5500a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:12.666423 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 960/1898] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:19:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:19:12.670878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3340e0c7-4c90-457f-860c-8f4b6f1ee6af None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:12.680306 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3340e0c7-4c90-457f-860c-8f4b6f1ee6af None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:12.685092 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338b238-e4c7-4e44-9389-446b9632fb4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:12.685691 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338b238-e4c7-4e44-9389-446b9632fb4d admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:12.685969 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338b238-e4c7-4e44-9389-446b9632fb4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:12.686267 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338b238-e4c7-4e44-9389-446b9632fb4d admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:12.686965 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6338b238-e4c7-4e44-9389-446b9632fb4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:12.688862 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6338b238-e4c7-4e44-9389-446b9632fb4d admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:12.718765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 939/1899] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:19:12 2022] POST /identity/v3/projects => generated 363 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:19:12.723570 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e3d144-dba6-4d9b-9574-a7fc43f5500a admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:12.737773 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e3d144-dba6-4d9b-9574-a7fc43f5500a admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:12.799992 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee43ce-49a9-4015-a22f-cd7720761b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:12.800630 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee43ce-49a9-4015-a22f-cd7720761b31 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:12.800910 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee43ce-49a9-4015-a22f-cd7720761b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:12.801183 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee43ce-49a9-4015-a22f-cd7720761b31 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:12.801821 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bee43ce-49a9-4015-a22f-cd7720761b31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:12.803798 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bee43ce-49a9-4015-a22f-cd7720761b31 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:12.826880 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 961/1900] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:19:12 2022] POST /identity/v3/users => generated 282 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:19:12.831164 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338b238-e4c7-4e44-9389-446b9632fb4d admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:12.841846 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338b238-e4c7-4e44-9389-446b9632fb4d admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:12.847595 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ed4b9d-4a1d-4248-9132-d952c6da3164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:12.848197 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed4b9d-4a1d-4248-9132-d952c6da3164 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:12.848476 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed4b9d-4a1d-4248-9132-d952c6da3164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:12.848744 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed4b9d-4a1d-4248-9132-d952c6da3164 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:12.849388 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ed4b9d-4a1d-4248-9132-d952c6da3164 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:12.851119 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ed4b9d-4a1d-4248-9132-d952c6da3164 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:12.861465 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 940/1901] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 05:19:12.865314 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee43ce-49a9-4015-a22f-cd7720761b31 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:12.875401 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee43ce-49a9-4015-a22f-cd7720761b31 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:12.881433 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35178725-8fd5-4c35-beb6-0d3cbfb7eed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:12.882061 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35178725-8fd5-4c35-beb6-0d3cbfb7eed0 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:12.882389 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35178725-8fd5-4c35-beb6-0d3cbfb7eed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:12.882631 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35178725-8fd5-4c35-beb6-0d3cbfb7eed0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:12.883254 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35178725-8fd5-4c35-beb6-0d3cbfb7eed0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:12.884927 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35178725-8fd5-4c35-beb6-0d3cbfb7eed0 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:12.895250 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 962/1902] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:19:12.899455 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ed4b9d-4a1d-4248-9132-d952c6da3164 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:12.910062 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ed4b9d-4a1d-4248-9132-d952c6da3164 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:12.916204 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444dd0fd-cec7-424b-b8a6-c2a0635eb094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:12.916858 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444dd0fd-cec7-424b-b8a6-c2a0635eb094 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:12.917167 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444dd0fd-cec7-424b-b8a6-c2a0635eb094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:12.917439 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444dd0fd-cec7-424b-b8a6-c2a0635eb094 admin admin] PATH_INFO: `/v3/projects/fa11798b1523442cb751dd532bd99f26/users/dccdc74d076d474eb5bfe1041e8d050a/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:12.934488 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444dd0fd-cec7-424b-b8a6-c2a0635eb094 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa11798b1523442cb751dd532bd99f26, user_id=dccdc74d076d474eb5bfe1041e8d050a, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:12.937192 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444dd0fd-cec7-424b-b8a6-c2a0635eb094 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:12.951294 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 941/1903] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:19:12 2022] PUT /identity/v3/projects/fa11798b1523442cb751dd532bd99f26/users/dccdc74d076d474eb5bfe1041e8d050a/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:19:12.961763 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32fe01-5442-4a75-bf6f-388f501a80bd None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:12.962078 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32fe01-5442-4a75-bf6f-388f501a80bd None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:12.962350 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32fe01-5442-4a75-bf6f-388f501a80bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:12.969796 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:12.993146 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-5f32fe01-5442-4a75-bf6f-388f501a80bd None None] MFA Rules not processed for user `dccdc74d076d474eb5bfe1041e8d050a`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:19:12.994457 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5f32fe01-5442-4a75-bf6f-388f501a80bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:13.064291 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 963/1904] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:19:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:19:13.068967 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444dd0fd-cec7-424b-b8a6-c2a0635eb094 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:13.133827 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444dd0fd-cec7-424b-b8a6-c2a0635eb094 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:13.139992 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f8db91-20a8-46d5-9b72-5f4b73d69371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:13.140741 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f8db91-20a8-46d5-9b72-5f4b73d69371 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:13.141089 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f8db91-20a8-46d5-9b72-5f4b73d69371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:13.141349 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f8db91-20a8-46d5-9b72-5f4b73d69371 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:13.142073 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f8db91-20a8-46d5-9b72-5f4b73d69371 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:13.144029 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f8db91-20a8-46d5-9b72-5f4b73d69371 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:13.173271 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 942/1905] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:19:13 2022] POST /identity/v3/projects => generated 367 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:19:13.177333 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32fe01-5442-4a75-bf6f-388f501a80bd None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:13.188291 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32fe01-5442-4a75-bf6f-388f501a80bd None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:13.245750 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7197d7b2-dcd2-4efb-a042-04ad3e0f8c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:13.246529 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197d7b2-dcd2-4efb-a042-04ad3e0f8c40 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:13.246804 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197d7b2-dcd2-4efb-a042-04ad3e0f8c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:13.247066 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197d7b2-dcd2-4efb-a042-04ad3e0f8c40 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:13.247694 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7197d7b2-dcd2-4efb-a042-04ad3e0f8c40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:13.249588 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7197d7b2-dcd2-4efb-a042-04ad3e0f8c40 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:13.271371 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 964/1906] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:19:13 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:19:13.275628 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f8db91-20a8-46d5-9b72-5f4b73d69371 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:13.286426 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f8db91-20a8-46d5-9b72-5f4b73d69371 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:13.291137 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376144e5-9c9e-427f-a234-49ae47391a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:13.291740 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376144e5-9c9e-427f-a234-49ae47391a90 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:13.292027 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376144e5-9c9e-427f-a234-49ae47391a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:13.292297 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376144e5-9c9e-427f-a234-49ae47391a90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:13.292945 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376144e5-9c9e-427f-a234-49ae47391a90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:13.294683 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376144e5-9c9e-427f-a234-49ae47391a90 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:13.305954 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 943/1907] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:19:13.309975 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7197d7b2-dcd2-4efb-a042-04ad3e0f8c40 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:13.320364 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7197d7b2-dcd2-4efb-a042-04ad3e0f8c40 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:13.325515 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867aee87-2a4d-418d-a85e-3183e0bfdebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:13.326223 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867aee87-2a4d-418d-a85e-3183e0bfdebb admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:13.326503 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867aee87-2a4d-418d-a85e-3183e0bfdebb admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:13.326778 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867aee87-2a4d-418d-a85e-3183e0bfdebb admin admin] PATH_INFO: `/v3/projects/29c9919a9e2940098ebcf8a33f3379c3/users/7c7733388e864a08803b68ca47a2e9b7/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:13.348838 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867aee87-2a4d-418d-a85e-3183e0bfdebb admin admin] RBAC: Authorizing `identity:create_grant(project_id=29c9919a9e2940098ebcf8a33f3379c3, user_id=7c7733388e864a08803b68ca47a2e9b7, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:13.351649 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867aee87-2a4d-418d-a85e-3183e0bfdebb admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:13.367178 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 965/1908] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:19:13 2022] PUT /identity/v3/projects/29c9919a9e2940098ebcf8a33f3379c3/users/7c7733388e864a08803b68ca47a2e9b7/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:19:13.371196 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376144e5-9c9e-427f-a234-49ae47391a90 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:13.381383 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376144e5-9c9e-427f-a234-49ae47391a90 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:13.427337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:13.441426 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1881e5-c1ea-4fb6-8721-120052e1d1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:13.442148 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1881e5-c1ea-4fb6-8721-120052e1d1d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:13.442436 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1881e5-c1ea-4fb6-8721-120052e1d1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:13.442730 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1881e5-c1ea-4fb6-8721-120052e1d1d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:13.443453 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1881e5-c1ea-4fb6-8721-120052e1d1d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:13.445357 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1881e5-c1ea-4fb6-8721-120052e1d1d0 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:13.456578 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 944/1909] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:19:13.461025 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867aee87-2a4d-418d-a85e-3183e0bfdebb admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:13.472030 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867aee87-2a4d-418d-a85e-3183e0bfdebb admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:13.477055 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c558e-0466-46ae-8a64-2ca76e4717fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:13.477675 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c558e-0466-46ae-8a64-2ca76e4717fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:13.477953 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c558e-0466-46ae-8a64-2ca76e4717fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:13.478265 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c558e-0466-46ae-8a64-2ca76e4717fd admin admin] PATH_INFO: `/v3/projects/29c9919a9e2940098ebcf8a33f3379c3/users/7c7733388e864a08803b68ca47a2e9b7/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:13.480201 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424c558e-0466-46ae-8a64-2ca76e4717fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=29c9919a9e2940098ebcf8a33f3379c3, user_id=7c7733388e864a08803b68ca47a2e9b7, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:13.482865 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424c558e-0466-46ae-8a64-2ca76e4717fd admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:13.494318 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 966/1910] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:19:13 2022] PUT /identity/v3/projects/29c9919a9e2940098ebcf8a33f3379c3/users/7c7733388e864a08803b68ca47a2e9b7/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:19:13.505982 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24aa0d-3b8d-45cc-b699-18a8ce000538 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:13.506298 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24aa0d-3b8d-45cc-b699-18a8ce000538 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:13.506597 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24aa0d-3b8d-45cc-b699-18a8ce000538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:13.538859 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-0e24aa0d-3b8d-45cc-b699-18a8ce000538 None None] MFA Rules not processed for user `7c7733388e864a08803b68ca47a2e9b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:19:13.540057 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-0e24aa0d-3b8d-45cc-b699-18a8ce000538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:13.605590 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 945/1911] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:19:13 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:19:13.621106 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c558e-0466-46ae-8a64-2ca76e4717fd admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:13.623461 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-424c558e-0466-46ae-8a64-2ca76e4717fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:13.676265 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:13.687621 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c558e-0466-46ae-8a64-2ca76e4717fd admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:13.692725 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac6593-be4c-4b51-bc6a-0564d961e1f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:13.693344 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac6593-be4c-4b51-bc6a-0564d961e1f3 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:13.693638 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac6593-be4c-4b51-bc6a-0564d961e1f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:13.693927 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac6593-be4c-4b51-bc6a-0564d961e1f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:13.703319 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ac6593-be4c-4b51-bc6a-0564d961e1f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:13.704742 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ac6593-be4c-4b51-bc6a-0564d961e1f3 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:13.714113 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 967/1912] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:19:13 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:19:13.800751 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24aa0d-3b8d-45cc-b699-18a8ce000538 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:13.810845 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24aa0d-3b8d-45cc-b699-18a8ce000538 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:13.815559 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14182e0-bf42-4389-b6df-7657090b8497 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:13.816164 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14182e0-bf42-4389-b6df-7657090b8497 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:13.816441 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14182e0-bf42-4389-b6df-7657090b8497 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:13.816705 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14182e0-bf42-4389-b6df-7657090b8497 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:13.881616 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14182e0-bf42-4389-b6df-7657090b8497 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:13.883694 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14182e0-bf42-4389-b6df-7657090b8497 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:13.895274 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 946/1913] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:19:13 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:19:17.759537 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac6593-be4c-4b51-bc6a-0564d961e1f3 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:17.762205 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-22ac6593-be4c-4b51-bc6a-0564d961e1f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:17.774300 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac6593-be4c-4b51-bc6a-0564d961e1f3 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:17.778412 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b6ce7-3e22-44be-b7c6-0ffd21082cfa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:17.779293 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b6ce7-3e22-44be-b7c6-0ffd21082cfa service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:17.779720 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b6ce7-3e22-44be-b7c6-0ffd21082cfa service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:17.780144 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b6ce7-3e22-44be-b7c6-0ffd21082cfa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:17.782810 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3d7b6ce7-3e22-44be-b7c6-0ffd21082cfa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:17.887645 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b6ce7-3e22-44be-b7c6-0ffd21082cfa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:17.889118 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b6ce7-3e22-44be-b7c6-0ffd21082cfa service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:17.915959 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 968/1914] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:19:17 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:19:23.890792 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=69 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:27.897181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=36 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:35.879001 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14182e0-bf42-4389-b6df-7657090b8497 service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:35.943955 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14182e0-bf42-4389-b6df-7657090b8497 service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:35.949429 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33009c-b890-415e-a6c8-12e4464d5f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:35.950124 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33009c-b890-415e-a6c8-12e4464d5f24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:35.950406 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33009c-b890-415e-a6c8-12e4464d5f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:35.950698 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33009c-b890-415e-a6c8-12e4464d5f24 admin admin] PATH_INFO: `/v3/users/dccdc74d076d474eb5bfe1041e8d050a` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:35.951991 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e33009c-b890-415e-a6c8-12e4464d5f24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dccdc74d076d474eb5bfe1041e8d050a)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:35.953979 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e33009c-b890-415e-a6c8-12e4464d5f24 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:36.002239 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9e33009c-b890-415e-a6c8-12e4464d5f24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dccdc74d076d474eb5bfe1041e8d050a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:36.009381 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9e33009c-b890-415e-a6c8-12e4464d5f24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dccdc74d076d474eb5bfe1041e8d050a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:36.010066 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9e33009c-b890-415e-a6c8-12e4464d5f24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dccdc74d076d474eb5bfe1041e8d050a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:36.019068 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9e33009c-b890-415e-a6c8-12e4464d5f24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dccdc74d076d474eb5bfe1041e8d050a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:36.019690 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9e33009c-b890-415e-a6c8-12e4464d5f24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dccdc74d076d474eb5bfe1041e8d050a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:36.036974 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 947/1915] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:19:35 2022] DELETE /identity/v3/users/dccdc74d076d474eb5bfe1041e8d050a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:19:36.041650 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f36477-610e-49b0-8adb-5398ba2e4d19 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:36.042130 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f36477-610e-49b0-8adb-5398ba2e4d19 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:36.042243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f36477-610e-49b0-8adb-5398ba2e4d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:36.061755 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-38f36477-610e-49b0-8adb-5398ba2e4d19 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:19:36.063041 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-38f36477-610e-49b0-8adb-5398ba2e4d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:36.124545 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 969/1916] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:19:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:19:36.134458 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33009c-b890-415e-a6c8-12e4464d5f24 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:36.135944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-9e33009c-b890-415e-a6c8-12e4464d5f24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:36.197696 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33009c-b890-415e-a6c8-12e4464d5f24 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:36.203749 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589a6db3-e187-446f-940f-3a86fd1d883e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:36.204460 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a6db3-e187-446f-940f-3a86fd1d883e service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:36.204681 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a6db3-e187-446f-940f-3a86fd1d883e service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:36.204950 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a6db3-e187-446f-940f-3a86fd1d883e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:36.216149 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589a6db3-e187-446f-940f-3a86fd1d883e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:36.219004 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589a6db3-e187-446f-940f-3a86fd1d883e service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:36.237914 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 948/1917] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:19:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:19:36.647461 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f36477-610e-49b0-8adb-5398ba2e4d19 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:36.654142 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-38f36477-610e-49b0-8adb-5398ba2e4d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:36.684052 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f36477-610e-49b0-8adb-5398ba2e4d19 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:36.688708 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44c3be8-cb1f-4c68-aad0-cf7f0afa607b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:36.689342 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c3be8-cb1f-4c68-aad0-cf7f0afa607b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:36.689621 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c3be8-cb1f-4c68-aad0-cf7f0afa607b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:36.689892 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c3be8-cb1f-4c68-aad0-cf7f0afa607b admin admin] PATH_INFO: `/v3/projects/fa11798b1523442cb751dd532bd99f26` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:36.691184 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44c3be8-cb1f-4c68-aad0-cf7f0afa607b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa11798b1523442cb751dd532bd99f26)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:36.693113 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44c3be8-cb1f-4c68-aad0-cf7f0afa607b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:36.744585 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d44c3be8-cb1f-4c68-aad0-cf7f0afa607b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa11798b1523442cb751dd532bd99f26', 'request_id': 'req-d44c3be8-cb1f-4c68-aad0-cf7f0afa607b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:36.756693 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d44c3be8-cb1f-4c68-aad0-cf7f0afa607b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa11798b1523442cb751dd532bd99f26', 'request_id': 'req-d44c3be8-cb1f-4c68-aad0-cf7f0afa607b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:36.764882 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d44c3be8-cb1f-4c68-aad0-cf7f0afa607b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa11798b1523442cb751dd532bd99f26', 'request_id': 'req-d44c3be8-cb1f-4c68-aad0-cf7f0afa607b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:36.775964 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d44c3be8-cb1f-4c68-aad0-cf7f0afa607b admin admin] The token cache is being invalidate because project fa11798b1523442cb751dd532bd99f26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:19:36.776249 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d44c3be8-cb1f-4c68-aad0-cf7f0afa607b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:36.776879 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d44c3be8-cb1f-4c68-aad0-cf7f0afa607b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:36.778438 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 970/1918] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:19:36 2022] DELETE /identity/v3/projects/fa11798b1523442cb751dd532bd99f26 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:19:36.782522 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589a6db3-e187-446f-940f-3a86fd1d883e service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:36.792518 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589a6db3-e187-446f-940f-3a86fd1d883e service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:36.810250 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:36.851828 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854f87d7-bf68-4a90-b601-b7acca4e2877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:36.852461 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f87d7-bf68-4a90-b601-b7acca4e2877 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:36.852738 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f87d7-bf68-4a90-b601-b7acca4e2877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:36.852994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f87d7-bf68-4a90-b601-b7acca4e2877 admin admin] PATH_INFO: `/v3/users/7c7733388e864a08803b68ca47a2e9b7` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:36.854148 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854f87d7-bf68-4a90-b601-b7acca4e2877 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c7733388e864a08803b68ca47a2e9b7)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:36.856097 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854f87d7-bf68-4a90-b601-b7acca4e2877 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:36.909656 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-854f87d7-bf68-4a90-b601-b7acca4e2877 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c7733388e864a08803b68ca47a2e9b7'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:36.917807 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-854f87d7-bf68-4a90-b601-b7acca4e2877 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c7733388e864a08803b68ca47a2e9b7'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:36.918834 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-854f87d7-bf68-4a90-b601-b7acca4e2877 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c7733388e864a08803b68ca47a2e9b7'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:36.926694 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-854f87d7-bf68-4a90-b601-b7acca4e2877 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c7733388e864a08803b68ca47a2e9b7'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:36.927508 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-854f87d7-bf68-4a90-b601-b7acca4e2877 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c7733388e864a08803b68ca47a2e9b7'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:36.947212 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 949/1919] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:19:36 2022] DELETE /identity/v3/users/7c7733388e864a08803b68ca47a2e9b7 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:19:37.224253 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44c3be8-cb1f-4c68-aad0-cf7f0afa607b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:37.225941 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d44c3be8-cb1f-4c68-aad0-cf7f0afa607b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:37.241759 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44c3be8-cb1f-4c68-aad0-cf7f0afa607b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:37.305262 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9368da1-fc02-4806-b9ee-fbf045ff21ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:37.305905 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9368da1-fc02-4806-b9ee-fbf045ff21ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:37.306215 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9368da1-fc02-4806-b9ee-fbf045ff21ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:37.306481 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9368da1-fc02-4806-b9ee-fbf045ff21ae admin admin] PATH_INFO: `/v3/projects/29c9919a9e2940098ebcf8a33f3379c3` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:37.307596 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9368da1-fc02-4806-b9ee-fbf045ff21ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=29c9919a9e2940098ebcf8a33f3379c3)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:37.309531 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9368da1-fc02-4806-b9ee-fbf045ff21ae admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:37.345772 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:37.362600 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b9368da1-fc02-4806-b9ee-fbf045ff21ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29c9919a9e2940098ebcf8a33f3379c3', 'request_id': 'req-b9368da1-fc02-4806-b9ee-fbf045ff21ae'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:37.368577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b9368da1-fc02-4806-b9ee-fbf045ff21ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29c9919a9e2940098ebcf8a33f3379c3', 'request_id': 'req-b9368da1-fc02-4806-b9ee-fbf045ff21ae'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:37.378586 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b9368da1-fc02-4806-b9ee-fbf045ff21ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29c9919a9e2940098ebcf8a33f3379c3', 'request_id': 'req-b9368da1-fc02-4806-b9ee-fbf045ff21ae'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:37.390331 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b9368da1-fc02-4806-b9ee-fbf045ff21ae admin admin] The token cache is being invalidate because project 29c9919a9e2940098ebcf8a33f3379c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:19:37.390608 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b9368da1-fc02-4806-b9ee-fbf045ff21ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:37.391383 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b9368da1-fc02-4806-b9ee-fbf045ff21ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:37.392880 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 971/1920] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:19:37 2022] DELETE /identity/v3/projects/29c9919a9e2940098ebcf8a33f3379c3 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:19:37.397885 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dff399-31ba-4efa-aaa7-6a344449b7e1 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:37.398209 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dff399-31ba-4efa-aaa7-6a344449b7e1 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:37.398486 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dff399-31ba-4efa-aaa7-6a344449b7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:37.419504 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-94dff399-31ba-4efa-aaa7-6a344449b7e1 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:19:37.420750 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-94dff399-31ba-4efa-aaa7-6a344449b7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:37.485937 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 950/1921] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:19:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:19:37.493567 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bde47e-64ac-4c43-9486-c55a93d02e87 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:37.493845 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bde47e-64ac-4c43-9486-c55a93d02e87 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:37.494272 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bde47e-64ac-4c43-9486-c55a93d02e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:37.515481 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d5bde47e-64ac-4c43-9486-c55a93d02e87 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:19:37.516853 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d5bde47e-64ac-4c43-9486-c55a93d02e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:37.531614 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 972/1922] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:19:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:19:37.532562 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dff399-31ba-4efa-aaa7-6a344449b7e1 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:37.544181 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dff399-31ba-4efa-aaa7-6a344449b7e1 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:37.549370 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04474b6-5680-4309-b323-2cdd564667f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:37.549987 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04474b6-5680-4309-b323-2cdd564667f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:37.550291 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04474b6-5680-4309-b323-2cdd564667f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:37.550558 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04474b6-5680-4309-b323-2cdd564667f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:37.551210 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04474b6-5680-4309-b323-2cdd564667f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:19:37.551655 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04474b6-5680-4309-b323-2cdd564667f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:37.553261 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04474b6-5680-4309-b323-2cdd564667f4 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:37.569972 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 951/1923] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:19:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:19:37.571271 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bde47e-64ac-4c43-9486-c55a93d02e87 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:37.585254 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bde47e-64ac-4c43-9486-c55a93d02e87 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:37.593123 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c041af7-be6f-4338-9663-567263997ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:37.593123 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c041af7-be6f-4338-9663-567263997ebd admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:37.593123 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c041af7-be6f-4338-9663-567263997ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:37.593123 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c041af7-be6f-4338-9663-567263997ebd admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:37.593554 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c041af7-be6f-4338-9663-567263997ebd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:37.595206 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c041af7-be6f-4338-9663-567263997ebd admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:37.638987 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 973/1924] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:19:37 2022] POST /identity/v3/projects => generated 371 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:19:37.643648 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04474b6-5680-4309-b323-2cdd564667f4 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:37.654706 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04474b6-5680-4309-b323-2cdd564667f4 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:37.681093 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:37.713662 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe0953c-1c6f-45c6-accd-cd1fd3b3278c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:37.714396 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0953c-1c6f-45c6-accd-cd1fd3b3278c admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:37.714672 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0953c-1c6f-45c6-accd-cd1fd3b3278c admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:37.714931 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0953c-1c6f-45c6-accd-cd1fd3b3278c admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:37.715576 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe0953c-1c6f-45c6-accd-cd1fd3b3278c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:37.717627 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe0953c-1c6f-45c6-accd-cd1fd3b3278c admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:37.744394 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 952/1925] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:19:37 2022] POST /identity/v3/users => generated 286 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:19:37.754228 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c041af7-be6f-4338-9663-567263997ebd admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:37.764938 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c041af7-be6f-4338-9663-567263997ebd admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:37.769660 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eddd878-f572-4be1-a148-d95ac1836871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:37.770300 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eddd878-f572-4be1-a148-d95ac1836871 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:37.770579 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eddd878-f572-4be1-a148-d95ac1836871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:37.770851 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eddd878-f572-4be1-a148-d95ac1836871 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:37.771498 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eddd878-f572-4be1-a148-d95ac1836871 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:37.773092 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eddd878-f572-4be1-a148-d95ac1836871 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:37.783823 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 974/1926] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:19:37.787910 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe0953c-1c6f-45c6-accd-cd1fd3b3278c admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:37.797987 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe0953c-1c6f-45c6-accd-cd1fd3b3278c admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:37.802760 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4ce41c-10a4-4a8e-9866-3a1a7885402f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:37.803363 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ce41c-10a4-4a8e-9866-3a1a7885402f admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:37.803635 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ce41c-10a4-4a8e-9866-3a1a7885402f admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:37.803901 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ce41c-10a4-4a8e-9866-3a1a7885402f admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:37.804543 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4ce41c-10a4-4a8e-9866-3a1a7885402f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:37.806184 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4ce41c-10a4-4a8e-9866-3a1a7885402f admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:37.816955 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 953/1927] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:19:37.821555 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eddd878-f572-4be1-a148-d95ac1836871 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:37.831951 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eddd878-f572-4be1-a148-d95ac1836871 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:37.836652 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2b4f37-3080-4fca-b918-cf25d3283d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:37.837310 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b4f37-3080-4fca-b918-cf25d3283d56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:37.837581 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b4f37-3080-4fca-b918-cf25d3283d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:37.837852 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b4f37-3080-4fca-b918-cf25d3283d56 admin admin] PATH_INFO: `/v3/projects/4d6f77efddfb480daaa7760f9c16e7d0/users/54825e5474ab439295a951eb0b45b68e/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:37.852586 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2b4f37-3080-4fca-b918-cf25d3283d56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d6f77efddfb480daaa7760f9c16e7d0, user_id=54825e5474ab439295a951eb0b45b68e, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:37.855279 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2b4f37-3080-4fca-b918-cf25d3283d56 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:37.869628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 975/1928] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:19:37 2022] PUT /identity/v3/projects/4d6f77efddfb480daaa7760f9c16e7d0/users/54825e5474ab439295a951eb0b45b68e/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:19:37.881168 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1c303-46e9-41ed-a596-4456341a93ba None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:37.881472 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1c303-46e9-41ed-a596-4456341a93ba None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:37.881737 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1c303-46e9-41ed-a596-4456341a93ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:37.913970 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-8fc1c303-46e9-41ed-a596-4456341a93ba None None] MFA Rules not processed for user `54825e5474ab439295a951eb0b45b68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:19:37.913970 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-8fc1c303-46e9-41ed-a596-4456341a93ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:37.976527 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 954/1929] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:19:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:19:37.988759 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2b4f37-3080-4fca-b918-cf25d3283d56 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:37.990490 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8a2b4f37-3080-4fca-b918-cf25d3283d56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:38.052906 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2b4f37-3080-4fca-b918-cf25d3283d56 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:38.056778 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da53bb8f-6bc7-4226-ad3f-b50a1463f45d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:38.057445 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53bb8f-6bc7-4226-ad3f-b50a1463f45d service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:38.057747 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53bb8f-6bc7-4226-ad3f-b50a1463f45d service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:38.058044 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53bb8f-6bc7-4226-ad3f-b50a1463f45d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:38.066700 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da53bb8f-6bc7-4226-ad3f-b50a1463f45d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:38.068129 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da53bb8f-6bc7-4226-ad3f-b50a1463f45d service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:38.077154 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 976/1930] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:19:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:19:38.098641 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1c303-46e9-41ed-a596-4456341a93ba None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:38.163124 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1c303-46e9-41ed-a596-4456341a93ba None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:38.169279 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7eab08-7ba0-4c30-9e1a-189a0811555d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:38.169915 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7eab08-7ba0-4c30-9e1a-189a0811555d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:38.170314 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7eab08-7ba0-4c30-9e1a-189a0811555d admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:38.170517 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7eab08-7ba0-4c30-9e1a-189a0811555d admin admin] PATH_INFO: `/v3/users/54825e5474ab439295a951eb0b45b68e` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:38.171821 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7eab08-7ba0-4c30-9e1a-189a0811555d admin admin] RBAC: Authorizing `identity:delete_user(user_id=54825e5474ab439295a951eb0b45b68e)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:38.173810 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7eab08-7ba0-4c30-9e1a-189a0811555d admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:38.217049 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:38.224331 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-da7eab08-7ba0-4c30-9e1a-189a0811555d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54825e5474ab439295a951eb0b45b68e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:38.231947 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-da7eab08-7ba0-4c30-9e1a-189a0811555d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54825e5474ab439295a951eb0b45b68e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:38.232637 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-da7eab08-7ba0-4c30-9e1a-189a0811555d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54825e5474ab439295a951eb0b45b68e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:38.240931 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-da7eab08-7ba0-4c30-9e1a-189a0811555d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54825e5474ab439295a951eb0b45b68e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:38.241544 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-da7eab08-7ba0-4c30-9e1a-189a0811555d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54825e5474ab439295a951eb0b45b68e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:38.259112 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 955/1931] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:19:38 2022] DELETE /identity/v3/users/54825e5474ab439295a951eb0b45b68e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:19:38.263819 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740e08b-5eb8-4dd4-a4dd-82bc91601fc7 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:38.264328 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740e08b-5eb8-4dd4-a4dd-82bc91601fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:38.264328 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740e08b-5eb8-4dd4-a4dd-82bc91601fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:38.283859 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f740e08b-5eb8-4dd4-a4dd-82bc91601fc7 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:19:38.285063 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f740e08b-5eb8-4dd4-a4dd-82bc91601fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:38.336896 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:38.351270 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 977/1932] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:19:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:19:38.361555 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7eab08-7ba0-4c30-9e1a-189a0811555d admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:38.363238 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-da7eab08-7ba0-4c30-9e1a-189a0811555d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:38.421519 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7eab08-7ba0-4c30-9e1a-189a0811555d admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:38.426036 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691fab2a-7054-4f6c-b69d-ec012d38eb12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:38.426648 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691fab2a-7054-4f6c-b69d-ec012d38eb12 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:38.426923 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691fab2a-7054-4f6c-b69d-ec012d38eb12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:38.427194 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691fab2a-7054-4f6c-b69d-ec012d38eb12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:38.436840 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691fab2a-7054-4f6c-b69d-ec012d38eb12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:38.438307 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691fab2a-7054-4f6c-b69d-ec012d38eb12 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:38.447878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 956/1933] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:19:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:19:38.741836 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740e08b-5eb8-4dd4-a4dd-82bc91601fc7 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:38.743526 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f740e08b-5eb8-4dd4-a4dd-82bc91601fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:38.773210 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740e08b-5eb8-4dd4-a4dd-82bc91601fc7 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:38.778733 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd506d2b-d78f-4bed-974b-5b2edb044b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:38.779354 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd506d2b-d78f-4bed-974b-5b2edb044b59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:38.779804 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd506d2b-d78f-4bed-974b-5b2edb044b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:38.780030 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd506d2b-d78f-4bed-974b-5b2edb044b59 admin admin] PATH_INFO: `/v3/projects/4d6f77efddfb480daaa7760f9c16e7d0` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:38.781317 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd506d2b-d78f-4bed-974b-5b2edb044b59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d6f77efddfb480daaa7760f9c16e7d0)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:38.783232 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd506d2b-d78f-4bed-974b-5b2edb044b59 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:38.834925 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-cd506d2b-d78f-4bed-974b-5b2edb044b59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d6f77efddfb480daaa7760f9c16e7d0', 'request_id': 'req-cd506d2b-d78f-4bed-974b-5b2edb044b59'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:38.841625 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-cd506d2b-d78f-4bed-974b-5b2edb044b59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d6f77efddfb480daaa7760f9c16e7d0', 'request_id': 'req-cd506d2b-d78f-4bed-974b-5b2edb044b59'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:38.849468 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-cd506d2b-d78f-4bed-974b-5b2edb044b59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d6f77efddfb480daaa7760f9c16e7d0', 'request_id': 'req-cd506d2b-d78f-4bed-974b-5b2edb044b59'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:38.862424 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-cd506d2b-d78f-4bed-974b-5b2edb044b59 admin admin] The token cache is being invalidate because project 4d6f77efddfb480daaa7760f9c16e7d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:19:38.862708 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-cd506d2b-d78f-4bed-974b-5b2edb044b59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:38.863409 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-cd506d2b-d78f-4bed-974b-5b2edb044b59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:38.865208 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 978/1934] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:19:38 2022] DELETE /identity/v3/projects/4d6f77efddfb480daaa7760f9c16e7d0 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:19:38.870315 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef2805-0f30-4e96-9ca3-f7e55e32d587 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:38.870867 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef2805-0f30-4e96-9ca3-f7e55e32d587 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:38.871179 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef2805-0f30-4e96-9ca3-f7e55e32d587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:38.891214 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-a7ef2805-0f30-4e96-9ca3-f7e55e32d587 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:19:38.892433 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a7ef2805-0f30-4e96-9ca3-f7e55e32d587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:38.960744 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 957/1935] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:19:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:19:38.968532 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91448c1-367e-4ff5-8105-97e9db90a0fd None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:38.968888 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91448c1-367e-4ff5-8105-97e9db90a0fd None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:38.969233 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91448c1-367e-4ff5-8105-97e9db90a0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:38.998500 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b91448c1-367e-4ff5-8105-97e9db90a0fd None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:19:38.999697 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b91448c1-367e-4ff5-8105-97e9db90a0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:39.008066 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 979/1936] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:19:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:19:39.012181 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ef2805-0f30-4e96-9ca3-f7e55e32d587 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:39.021742 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ef2805-0f30-4e96-9ca3-f7e55e32d587 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:39.034625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963fecd3-6112-4d64-9b0a-6df91293a0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:39.035252 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963fecd3-6112-4d64-9b0a-6df91293a0df admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:39.035524 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963fecd3-6112-4d64-9b0a-6df91293a0df admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:39.035802 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963fecd3-6112-4d64-9b0a-6df91293a0df admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:39.036429 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963fecd3-6112-4d64-9b0a-6df91293a0df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:19:39.036762 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963fecd3-6112-4d64-9b0a-6df91293a0df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:39.038401 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963fecd3-6112-4d64-9b0a-6df91293a0df admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:39.052623 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 958/1937] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:19:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:19:39.056979 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91448c1-367e-4ff5-8105-97e9db90a0fd None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:39.066638 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91448c1-367e-4ff5-8105-97e9db90a0fd None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:39.071488 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be1be0b-f0f8-4347-8c3d-1dba4d5e0f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:39.072124 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1be0b-f0f8-4347-8c3d-1dba4d5e0f04 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:39.072402 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1be0b-f0f8-4347-8c3d-1dba4d5e0f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:39.072670 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1be0b-f0f8-4347-8c3d-1dba4d5e0f04 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:39.073348 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be1be0b-f0f8-4347-8c3d-1dba4d5e0f04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:39.075326 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be1be0b-f0f8-4347-8c3d-1dba4d5e0f04 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:39.107028 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 980/1938] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:19:39 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:19:39.108441 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963fecd3-6112-4d64-9b0a-6df91293a0df admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:39.118603 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963fecd3-6112-4d64-9b0a-6df91293a0df admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:39.164290 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:39.178519 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77640c2-4d1b-46e0-9209-828eff6ce60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:39.179127 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77640c2-4d1b-46e0-9209-828eff6ce60e admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:39.179400 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77640c2-4d1b-46e0-9209-828eff6ce60e admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:39.179670 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77640c2-4d1b-46e0-9209-828eff6ce60e admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:39.180284 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77640c2-4d1b-46e0-9209-828eff6ce60e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:39.182233 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77640c2-4d1b-46e0-9209-828eff6ce60e admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:39.203868 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 959/1939] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:19:39 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:19:39.208178 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be1be0b-f0f8-4347-8c3d-1dba4d5e0f04 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:39.218745 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be1be0b-f0f8-4347-8c3d-1dba4d5e0f04 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:39.224205 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56069f6-6257-4d49-8dc6-17de077c6e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:39.224811 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56069f6-6257-4d49-8dc6-17de077c6e25 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:39.225087 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56069f6-6257-4d49-8dc6-17de077c6e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:39.225352 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56069f6-6257-4d49-8dc6-17de077c6e25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:39.226028 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56069f6-6257-4d49-8dc6-17de077c6e25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:39.227630 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56069f6-6257-4d49-8dc6-17de077c6e25 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:39.238435 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 981/1940] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:19:39.242544 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77640c2-4d1b-46e0-9209-828eff6ce60e admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:39.252973 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77640c2-4d1b-46e0-9209-828eff6ce60e admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:39.258256 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f331ab65-bd4f-4887-9824-87bacf312f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:39.258857 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331ab65-bd4f-4887-9824-87bacf312f2e admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:39.259135 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331ab65-bd4f-4887-9824-87bacf312f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:39.259403 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331ab65-bd4f-4887-9824-87bacf312f2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:39.260048 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f331ab65-bd4f-4887-9824-87bacf312f2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:39.261658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f331ab65-bd4f-4887-9824-87bacf312f2e admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:39.273519 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 960/1941] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 05:19:39.277748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56069f6-6257-4d49-8dc6-17de077c6e25 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:39.288669 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56069f6-6257-4d49-8dc6-17de077c6e25 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:39.294313 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b834822-58b4-40d0-9ff2-d504757344a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:39.294962 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b834822-58b4-40d0-9ff2-d504757344a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:39.295237 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b834822-58b4-40d0-9ff2-d504757344a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:39.295498 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b834822-58b4-40d0-9ff2-d504757344a2 admin admin] PATH_INFO: `/v3/projects/9c1aacb43c5f432da847bfdd957aaedc/users/9db97a303943436e9c8b6b6c2417d5c5/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:39.314374 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b834822-58b4-40d0-9ff2-d504757344a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c1aacb43c5f432da847bfdd957aaedc, user_id=9db97a303943436e9c8b6b6c2417d5c5, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:39.317053 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b834822-58b4-40d0-9ff2-d504757344a2 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:39.335962 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 982/1942] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:19:39 2022] PUT /identity/v3/projects/9c1aacb43c5f432da847bfdd957aaedc/users/9db97a303943436e9c8b6b6c2417d5c5/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:19:39.346855 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989df3c7-a234-4341-839e-292c92472103 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:39.347080 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989df3c7-a234-4341-839e-292c92472103 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:39.347343 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989df3c7-a234-4341-839e-292c92472103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:39.377219 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-989df3c7-a234-4341-839e-292c92472103 None None] MFA Rules not processed for user `9db97a303943436e9c8b6b6c2417d5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:19:39.378454 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-989df3c7-a234-4341-839e-292c92472103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:39.445066 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 961/1943] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:19:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:19:39.456308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b834822-58b4-40d0-9ff2-d504757344a2 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:39.458317 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1b834822-58b4-40d0-9ff2-d504757344a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:39.517348 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b834822-58b4-40d0-9ff2-d504757344a2 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:39.521593 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a61e00-8887-4be3-8c76-5c7b355c56d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:39.522191 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a61e00-8887-4be3-8c76-5c7b355c56d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:39.522485 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a61e00-8887-4be3-8c76-5c7b355c56d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:39.522748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a61e00-8887-4be3-8c76-5c7b355c56d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:39.532961 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a61e00-8887-4be3-8c76-5c7b355c56d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:39.534446 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a61e00-8887-4be3-8c76-5c7b355c56d2 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:39.543362 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 983/1944] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:19:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:19:42.144896 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06e07d-0348-4672-9d5e-47472a6c08b9 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:42.145401 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06e07d-0348-4672-9d5e-47472a6c08b9 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:42.145662 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06e07d-0348-4672-9d5e-47472a6c08b9 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:42.147560 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 962/1945] 10.222.0.36 () {54 vars in 921 bytes} [Wed Aug 31 05:19:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:19:42.159039 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8ef1a-6d6e-4213-9981-42f7dfb55a9d None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:42.159481 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8ef1a-6d6e-4213-9981-42f7dfb55a9d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:42.159714 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8ef1a-6d6e-4213-9981-42f7dfb55a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:42.184650 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-32b8ef1a-6d6e-4213-9981-42f7dfb55a9d None None] MFA Rules not processed for user `f27665b1574a4db7b7a9771f7d775826`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:19:42.186204 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-32b8ef1a-6d6e-4213-9981-42f7dfb55a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:42.196590 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:42.286163 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 984/1946] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Aug 31 05:19:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:19:42.298625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06e07d-0348-4672-9d5e-47472a6c08b9 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:42.301007 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-6c06e07d-0348-4672-9d5e-47472a6c08b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:42.333092 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06e07d-0348-4672-9d5e-47472a6c08b9 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:42.350274 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade82777-51be-4c08-9697-ceeedcc211ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:42.350938 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade82777-51be-4c08-9697-ceeedcc211ab service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:42.351235 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade82777-51be-4c08-9697-ceeedcc211ab service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:42.351504 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade82777-51be-4c08-9697-ceeedcc211ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:42.361766 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade82777-51be-4c08-9697-ceeedcc211ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:42.363801 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade82777-51be-4c08-9697-ceeedcc211ab service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:42.376444 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 963/1947] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:19:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 05:19:44.050766 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b8ef1a-6d6e-4213-9981-42f7dfb55a9d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:44.159579 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b8ef1a-6d6e-4213-9981-42f7dfb55a9d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:44.170369 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f7e73b-bd0d-4786-8db1-18f12e781e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:44.172185 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7e73b-bd0d-4786-8db1-18f12e781e3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:44.172779 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7e73b-bd0d-4786-8db1-18f12e781e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:44.173632 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7e73b-bd0d-4786-8db1-18f12e781e3d admin admin] PATH_INFO: `/v3/users/9db97a303943436e9c8b6b6c2417d5c5` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:44.175329 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f7e73b-bd0d-4786-8db1-18f12e781e3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9db97a303943436e9c8b6b6c2417d5c5)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:44.177916 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f7e73b-bd0d-4786-8db1-18f12e781e3d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:44.245432 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a5f7e73b-bd0d-4786-8db1-18f12e781e3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9db97a303943436e9c8b6b6c2417d5c5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:44.253804 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a5f7e73b-bd0d-4786-8db1-18f12e781e3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9db97a303943436e9c8b6b6c2417d5c5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:44.267917 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a5f7e73b-bd0d-4786-8db1-18f12e781e3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9db97a303943436e9c8b6b6c2417d5c5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:44.268877 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a5f7e73b-bd0d-4786-8db1-18f12e781e3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9db97a303943436e9c8b6b6c2417d5c5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:44.283568 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a5f7e73b-bd0d-4786-8db1-18f12e781e3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9db97a303943436e9c8b6b6c2417d5c5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:44.292986 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 985/1948] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:19:44 2022] DELETE /identity/v3/users/9db97a303943436e9c8b6b6c2417d5c5 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:19:44.295302 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096716bf-9e9f-431a-8d15-6a0b9c73df92 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:44.295779 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096716bf-9e9f-431a-8d15-6a0b9c73df92 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:44.295878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096716bf-9e9f-431a-8d15-6a0b9c73df92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:44.315002 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-096716bf-9e9f-431a-8d15-6a0b9c73df92 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:19:44.316212 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-096716bf-9e9f-431a-8d15-6a0b9c73df92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:44.399030 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 964/1949] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:19:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:19:44.409493 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f7e73b-bd0d-4786-8db1-18f12e781e3d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:44.411051 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a5f7e73b-bd0d-4786-8db1-18f12e781e3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:44.441470 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:44.469744 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f7e73b-bd0d-4786-8db1-18f12e781e3d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:44.473691 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1be947-4f49-4f73-a027-7355e966b5c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:44.474356 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1be947-4f49-4f73-a027-7355e966b5c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:44.474626 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1be947-4f49-4f73-a027-7355e966b5c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:44.474888 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1be947-4f49-4f73-a027-7355e966b5c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:44.484869 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1be947-4f49-4f73-a027-7355e966b5c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:44.486648 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1be947-4f49-4f73-a027-7355e966b5c5 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:44.499188 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 986/1950] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:19:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:19:44.811649 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096716bf-9e9f-431a-8d15-6a0b9c73df92 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:44.813309 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-096716bf-9e9f-431a-8d15-6a0b9c73df92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:44.826087 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096716bf-9e9f-431a-8d15-6a0b9c73df92 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:44.831437 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1e79b0-c38d-44d1-a12a-84cec988f535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:44.832324 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e79b0-c38d-44d1-a12a-84cec988f535 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:44.832747 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e79b0-c38d-44d1-a12a-84cec988f535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:44.833121 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e79b0-c38d-44d1-a12a-84cec988f535 admin admin] PATH_INFO: `/v3/projects/9c1aacb43c5f432da847bfdd957aaedc` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:44.834506 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1e79b0-c38d-44d1-a12a-84cec988f535 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c1aacb43c5f432da847bfdd957aaedc)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:44.837417 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1e79b0-c38d-44d1-a12a-84cec988f535 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:44.883432 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:44.894769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-1e1e79b0-c38d-44d1-a12a-84cec988f535 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c1aacb43c5f432da847bfdd957aaedc', 'request_id': 'req-1e1e79b0-c38d-44d1-a12a-84cec988f535'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:44.901789 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-1e1e79b0-c38d-44d1-a12a-84cec988f535 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c1aacb43c5f432da847bfdd957aaedc', 'request_id': 'req-1e1e79b0-c38d-44d1-a12a-84cec988f535'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:44.910291 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-1e1e79b0-c38d-44d1-a12a-84cec988f535 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c1aacb43c5f432da847bfdd957aaedc', 'request_id': 'req-1e1e79b0-c38d-44d1-a12a-84cec988f535'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:44.925208 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-1e1e79b0-c38d-44d1-a12a-84cec988f535 admin admin] The token cache is being invalidate because project 9c1aacb43c5f432da847bfdd957aaedc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:19:44.925561 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-1e1e79b0-c38d-44d1-a12a-84cec988f535 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:44.926281 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-1e1e79b0-c38d-44d1-a12a-84cec988f535 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:19:44.928539 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 965/1951] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:19:44 2022] DELETE /identity/v3/projects/9c1aacb43c5f432da847bfdd957aaedc => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:19:44.935190 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c64d5-dde4-4652-84d0-c07cc4a9ad35 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:44.935574 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c64d5-dde4-4652-84d0-c07cc4a9ad35 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:44.935931 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c64d5-dde4-4652-84d0-c07cc4a9ad35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:44.958040 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-5f5c64d5-dde4-4652-84d0-c07cc4a9ad35 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:19:44.959370 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5f5c64d5-dde4-4652-84d0-c07cc4a9ad35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:45.038844 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 987/1952] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:19:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:19:45.048935 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b5888-904c-45dc-968c-9e0e5907f025 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:45.049420 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b5888-904c-45dc-968c-9e0e5907f025 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:45.049749 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b5888-904c-45dc-968c-9e0e5907f025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:45.081730 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-036b5888-904c-45dc-968c-9e0e5907f025 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:19:45.082994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-036b5888-904c-45dc-968c-9e0e5907f025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:45.097995 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c64d5-dde4-4652-84d0-c07cc4a9ad35 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:45.104031 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 966/1953] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:19:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:19:45.114382 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c64d5-dde4-4652-84d0-c07cc4a9ad35 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:45.119034 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f4470f-7178-45fa-8619-4a48117b26e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:45.119894 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4470f-7178-45fa-8619-4a48117b26e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:45.120236 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4470f-7178-45fa-8619-4a48117b26e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:45.120631 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4470f-7178-45fa-8619-4a48117b26e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:45.121292 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f4470f-7178-45fa-8619-4a48117b26e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:19:45.121746 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f4470f-7178-45fa-8619-4a48117b26e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:45.123540 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f4470f-7178-45fa-8619-4a48117b26e0 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:45.140878 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 988/1954] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:19:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:19:45.141737 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b5888-904c-45dc-968c-9e0e5907f025 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:45.152800 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b5888-904c-45dc-968c-9e0e5907f025 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:45.161307 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e82b204-6efc-42f8-974d-235056690a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:45.161941 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e82b204-6efc-42f8-974d-235056690a93 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:45.162413 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e82b204-6efc-42f8-974d-235056690a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:45.162688 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e82b204-6efc-42f8-974d-235056690a93 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:45.163342 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e82b204-6efc-42f8-974d-235056690a93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:45.166056 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e82b204-6efc-42f8-974d-235056690a93 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:45.200077 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 967/1955] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:19:45 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:19:45.204140 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f4470f-7178-45fa-8619-4a48117b26e0 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:45.215386 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f4470f-7178-45fa-8619-4a48117b26e0 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:45.275661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275036fe-5aa7-4413-b309-618272375d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:45.276199 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275036fe-5aa7-4413-b309-618272375d36 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:45.276602 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275036fe-5aa7-4413-b309-618272375d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:45.276776 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275036fe-5aa7-4413-b309-618272375d36 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:45.277412 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275036fe-5aa7-4413-b309-618272375d36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:45.279534 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275036fe-5aa7-4413-b309-618272375d36 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:45.304783 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 989/1956] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:19:45 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:19:45.309034 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e82b204-6efc-42f8-974d-235056690a93 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:45.321967 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e82b204-6efc-42f8-974d-235056690a93 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:45.327216 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f72e21-f8cc-4273-8f6f-f36ca52a3737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:45.328118 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f72e21-f8cc-4273-8f6f-f36ca52a3737 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:45.328593 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f72e21-f8cc-4273-8f6f-f36ca52a3737 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:45.329077 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f72e21-f8cc-4273-8f6f-f36ca52a3737 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:45.329997 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f72e21-f8cc-4273-8f6f-f36ca52a3737 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:45.331733 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f72e21-f8cc-4273-8f6f-f36ca52a3737 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:45.343170 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 968/1957] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:19:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:19:45.347246 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275036fe-5aa7-4413-b309-618272375d36 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:45.357699 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275036fe-5aa7-4413-b309-618272375d36 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:45.362978 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39877d24-9f38-423e-b423-cb8ae3de5643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:45.363606 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39877d24-9f38-423e-b423-cb8ae3de5643 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:45.363883 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39877d24-9f38-423e-b423-cb8ae3de5643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:45.364158 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39877d24-9f38-423e-b423-cb8ae3de5643 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:45.364865 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39877d24-9f38-423e-b423-cb8ae3de5643 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:45.366533 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39877d24-9f38-423e-b423-cb8ae3de5643 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:45.377721 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 990/1958] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:19:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:19:45.381966 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f72e21-f8cc-4273-8f6f-f36ca52a3737 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:45.392534 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f72e21-f8cc-4273-8f6f-f36ca52a3737 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:45.397263 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d636e5b-2f4e-44ba-a82a-4e90e04fc2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:45.397927 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d636e5b-2f4e-44ba-a82a-4e90e04fc2f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:45.398291 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d636e5b-2f4e-44ba-a82a-4e90e04fc2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:45.398567 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d636e5b-2f4e-44ba-a82a-4e90e04fc2f4 admin admin] PATH_INFO: `/v3/projects/ae98273d8d0040b39ed83c8b7aee74a6/users/a0063b9fa5dc4c3789db536f99a211fc/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:45.414853 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d636e5b-2f4e-44ba-a82a-4e90e04fc2f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae98273d8d0040b39ed83c8b7aee74a6, user_id=a0063b9fa5dc4c3789db536f99a211fc, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:45.417661 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d636e5b-2f4e-44ba-a82a-4e90e04fc2f4 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:45.431575 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 969/1959] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:19:45 2022] PUT /identity/v3/projects/ae98273d8d0040b39ed83c8b7aee74a6/users/a0063b9fa5dc4c3789db536f99a211fc/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:19:45.436894 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef07c9-8c7e-4a18-866e-98ae51444f5a None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:45.437178 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef07c9-8c7e-4a18-866e-98ae51444f5a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:45.437482 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef07c9-8c7e-4a18-866e-98ae51444f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:45.456408 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-2bef07c9-8c7e-4a18-866e-98ae51444f5a None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:19:45.457591 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2bef07c9-8c7e-4a18-866e-98ae51444f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:45.487488 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:45.520053 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 991/1960] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:19:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:19:45.530024 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d636e5b-2f4e-44ba-a82a-4e90e04fc2f4 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:45.532128 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-1d636e5b-2f4e-44ba-a82a-4e90e04fc2f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:45.590387 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d636e5b-2f4e-44ba-a82a-4e90e04fc2f4 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:45.595084 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1ab2a-887f-43e7-a76b-bd981dac6569 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:45.595698 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1ab2a-887f-43e7-a76b-bd981dac6569 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:45.595972 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1ab2a-887f-43e7-a76b-bd981dac6569 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:45.596243 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1ab2a-887f-43e7-a76b-bd981dac6569 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:45.608163 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e1ab2a-887f-43e7-a76b-bd981dac6569 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:45.609652 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e1ab2a-887f-43e7-a76b-bd981dac6569 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:45.620279 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 970/1961] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:19:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:19:50.834733 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326263a3-e620-4f5b-bdeb-c0e008e749e6 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:50.835645 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326263a3-e620-4f5b-bdeb-c0e008e749e6 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:50.836348 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326263a3-e620-4f5b-bdeb-c0e008e749e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:50.885427 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-326263a3-e620-4f5b-bdeb-c0e008e749e6 None None] MFA Rules not processed for user `a0063b9fa5dc4c3789db536f99a211fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:19:50.888892 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-326263a3-e620-4f5b-bdeb-c0e008e749e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:51.001336 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 992/1962] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:19:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:19:51.005396 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1ab2a-887f-43e7-a76b-bd981dac6569 service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:51.015170 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1ab2a-887f-43e7-a76b-bd981dac6569 service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:51.023579 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a615aa5-f7ff-4f9e-ae69-8dfa214f9771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:51.024354 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a615aa5-f7ff-4f9e-ae69-8dfa214f9771 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:51.024700 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a615aa5-f7ff-4f9e-ae69-8dfa214f9771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:51.025679 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a615aa5-f7ff-4f9e-ae69-8dfa214f9771 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:51.027126 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a615aa5-f7ff-4f9e-ae69-8dfa214f9771 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:51.030416 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a615aa5-f7ff-4f9e-ae69-8dfa214f9771 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:51.086079 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 971/1963] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:19:51 2022] POST /identity/v3/projects => generated 373 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:19:51.087621 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326263a3-e620-4f5b-bdeb-c0e008e749e6 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:51.099826 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326263a3-e620-4f5b-bdeb-c0e008e749e6 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:51.173299 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b392d12-7ce8-46fa-ae9f-f192f243ed0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:51.173911 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b392d12-7ce8-46fa-ae9f-f192f243ed0e admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:51.174220 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b392d12-7ce8-46fa-ae9f-f192f243ed0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:51.174488 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b392d12-7ce8-46fa-ae9f-f192f243ed0e admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:51.175128 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b392d12-7ce8-46fa-ae9f-f192f243ed0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:51.177022 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b392d12-7ce8-46fa-ae9f-f192f243ed0e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:51.202122 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 993/1964] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:19:51 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:19:51.205125 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a615aa5-f7ff-4f9e-ae69-8dfa214f9771 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:51.215500 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a615aa5-f7ff-4f9e-ae69-8dfa214f9771 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:51.220916 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb8aa3-8a1f-4049-b0e9-8078f3ec0834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:51.221737 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb8aa3-8a1f-4049-b0e9-8078f3ec0834 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:51.222032 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb8aa3-8a1f-4049-b0e9-8078f3ec0834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:51.222431 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb8aa3-8a1f-4049-b0e9-8078f3ec0834 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:51.223086 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eb8aa3-8a1f-4049-b0e9-8078f3ec0834 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:51.224714 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eb8aa3-8a1f-4049-b0e9-8078f3ec0834 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:51.236117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 972/1965] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:19:51.240460 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b392d12-7ce8-46fa-ae9f-f192f243ed0e admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:51.251305 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b392d12-7ce8-46fa-ae9f-f192f243ed0e admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:51.256171 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83147b80-66f4-40d9-a266-69d69044ad71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:51.256830 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83147b80-66f4-40d9-a266-69d69044ad71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:51.257103 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83147b80-66f4-40d9-a266-69d69044ad71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:51.257371 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83147b80-66f4-40d9-a266-69d69044ad71 admin admin] PATH_INFO: `/v3/projects/3ba29a5baed640e599b49e898ac5a1cf/users/1228fd3ec0df43ed8723d95ff00cbf5d/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:51.273960 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83147b80-66f4-40d9-a266-69d69044ad71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ba29a5baed640e599b49e898ac5a1cf, user_id=1228fd3ec0df43ed8723d95ff00cbf5d, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:51.276631 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83147b80-66f4-40d9-a266-69d69044ad71 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:51.292107 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 994/1966] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:19:51 2022] PUT /identity/v3/projects/3ba29a5baed640e599b49e898ac5a1cf/users/1228fd3ec0df43ed8723d95ff00cbf5d/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:19:51.296948 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb8aa3-8a1f-4049-b0e9-8078f3ec0834 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:51.311490 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb8aa3-8a1f-4049-b0e9-8078f3ec0834 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:51.357121 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:51.390382 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d09a6e-51bc-4b7a-8c68-fb3421d72eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:51.391083 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d09a6e-51bc-4b7a-8c68-fb3421d72eee admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:51.391416 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d09a6e-51bc-4b7a-8c68-fb3421d72eee admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:51.391684 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d09a6e-51bc-4b7a-8c68-fb3421d72eee admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:51.392331 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d09a6e-51bc-4b7a-8c68-fb3421d72eee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:51.393928 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d09a6e-51bc-4b7a-8c68-fb3421d72eee admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:51.406056 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 973/1967] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:19:51.409046 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83147b80-66f4-40d9-a266-69d69044ad71 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:51.419490 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83147b80-66f4-40d9-a266-69d69044ad71 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:51.424394 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6a79ec-b796-47ff-b500-ebcbbb5784a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:51.425161 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a79ec-b796-47ff-b500-ebcbbb5784a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:51.425434 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a79ec-b796-47ff-b500-ebcbbb5784a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:51.425706 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a79ec-b796-47ff-b500-ebcbbb5784a0 admin admin] PATH_INFO: `/v3/projects/3ba29a5baed640e599b49e898ac5a1cf/users/1228fd3ec0df43ed8723d95ff00cbf5d/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:51.427695 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6a79ec-b796-47ff-b500-ebcbbb5784a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ba29a5baed640e599b49e898ac5a1cf, user_id=1228fd3ec0df43ed8723d95ff00cbf5d, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:51.430445 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6a79ec-b796-47ff-b500-ebcbbb5784a0 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:51.441635 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 995/1968] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:19:51 2022] PUT /identity/v3/projects/3ba29a5baed640e599b49e898ac5a1cf/users/1228fd3ec0df43ed8723d95ff00cbf5d/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:19:56.967353 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab96b2-1cbf-4395-b35d-beda75f14cec None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:56.968024 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab96b2-1cbf-4395-b35d-beda75f14cec None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:56.968024 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab96b2-1cbf-4395-b35d-beda75f14cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:57.017900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-a0ab96b2-1cbf-4395-b35d-beda75f14cec None None] MFA Rules not processed for user `1228fd3ec0df43ed8723d95ff00cbf5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:19:57.019091 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a0ab96b2-1cbf-4395-b35d-beda75f14cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:57.106488 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 974/1969] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:19:56 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:19:57.120785 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6a79ec-b796-47ff-b500-ebcbbb5784a0 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:19:57.123080 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ac6a79ec-b796-47ff-b500-ebcbbb5784a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:57.146468 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:19:57.211337 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6a79ec-b796-47ff-b500-ebcbbb5784a0 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:19:57.217042 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb644d-ae41-4e34-bce9-527b4bc3279b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:19:57.217689 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb644d-ae41-4e34-bce9-527b4bc3279b service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:57.217966 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb644d-ae41-4e34-bce9-527b4bc3279b service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:57.218292 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb644d-ae41-4e34-bce9-527b4bc3279b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:57.286948 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cb644d-ae41-4e34-bce9-527b4bc3279b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:19:57.288593 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cb644d-ae41-4e34-bce9-527b4bc3279b service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:19:57.306137 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 996/1970] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:19:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:19:58.421555 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ed894-5dd5-4518-9b67-44694495e376 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:19:58.422508 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ed894-5dd5-4518-9b67-44694495e376 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:19:58.422818 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ed894-5dd5-4518-9b67-44694495e376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:19:58.457662 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-075ed894-5dd5-4518-9b67-44694495e376 None None] MFA Rules not processed for user `a0063b9fa5dc4c3789db536f99a211fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:19:58.462915 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-075ed894-5dd5-4518-9b67-44694495e376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:19:58.473572 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 975/1971] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:19:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:03.157611 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb644d-ae41-4e34-bce9-527b4bc3279b service nova] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:03.160104 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-13cb644d-ae41-4e34-bce9-527b4bc3279b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:03.267838 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb644d-ae41-4e34-bce9-527b4bc3279b service nova] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:03.272989 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b373bf-eb04-4946-91e5-3687cfdf78c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:03.273707 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b373bf-eb04-4946-91e5-3687cfdf78c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:03.274075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b373bf-eb04-4946-91e5-3687cfdf78c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:03.274408 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b373bf-eb04-4946-91e5-3687cfdf78c8 admin admin] PATH_INFO: `/v3/users/57fd6909d759418ab043936625d0680e` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:03.275592 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b373bf-eb04-4946-91e5-3687cfdf78c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57fd6909d759418ab043936625d0680e)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:03.277751 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b373bf-eb04-4946-91e5-3687cfdf78c8 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:03.333869 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-74b373bf-eb04-4946-91e5-3687cfdf78c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57fd6909d759418ab043936625d0680e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:03.347067 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-74b373bf-eb04-4946-91e5-3687cfdf78c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57fd6909d759418ab043936625d0680e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:03.350810 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:03.361056 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-74b373bf-eb04-4946-91e5-3687cfdf78c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57fd6909d759418ab043936625d0680e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:03.362189 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-74b373bf-eb04-4946-91e5-3687cfdf78c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57fd6909d759418ab043936625d0680e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:03.376838 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-74b373bf-eb04-4946-91e5-3687cfdf78c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57fd6909d759418ab043936625d0680e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:03.386872 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 997/1972] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:20:03 2022] DELETE /identity/v3/users/57fd6909d759418ab043936625d0680e => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:03.393064 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046781a-5a82-40cf-8377-1e96307cba2f None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:03.393630 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046781a-5a82-40cf-8377-1e96307cba2f None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:03.394051 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046781a-5a82-40cf-8377-1e96307cba2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:03.429391 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-8046781a-5a82-40cf-8377-1e96307cba2f None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:03.431914 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-8046781a-5a82-40cf-8377-1e96307cba2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:03.529723 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 976/1973] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:03.545927 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b373bf-eb04-4946-91e5-3687cfdf78c8 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:03.549223 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-74b373bf-eb04-4946-91e5-3687cfdf78c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:03.613653 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b373bf-eb04-4946-91e5-3687cfdf78c8 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:03.617258 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad91138-a570-44d0-9095-2e949668cc8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:03.617879 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad91138-a570-44d0-9095-2e949668cc8c service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:03.618182 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad91138-a570-44d0-9095-2e949668cc8c service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:03.618451 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad91138-a570-44d0-9095-2e949668cc8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:03.627733 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad91138-a570-44d0-9095-2e949668cc8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:03.629169 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad91138-a570-44d0-9095-2e949668cc8c service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:03.643557 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 998/1974] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:20:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:20:04.079076 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8046781a-5a82-40cf-8377-1e96307cba2f None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:04.080802 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-8046781a-5a82-40cf-8377-1e96307cba2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:04.099962 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8046781a-5a82-40cf-8377-1e96307cba2f None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:04.117705 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2925250f-8eae-43da-905f-8d3e22de4831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:04.118396 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925250f-8eae-43da-905f-8d3e22de4831 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:04.118673 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925250f-8eae-43da-905f-8d3e22de4831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:04.118969 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925250f-8eae-43da-905f-8d3e22de4831 admin admin] PATH_INFO: `/v3/projects/2252d9e0337446b88dc9d3c929683b62` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:04.120895 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2925250f-8eae-43da-905f-8d3e22de4831 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2252d9e0337446b88dc9d3c929683b62)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:04.122977 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2925250f-8eae-43da-905f-8d3e22de4831 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:04.232363 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:04.256245 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-2925250f-8eae-43da-905f-8d3e22de4831 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2252d9e0337446b88dc9d3c929683b62', 'request_id': 'req-2925250f-8eae-43da-905f-8d3e22de4831'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:04.271844 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-2925250f-8eae-43da-905f-8d3e22de4831 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2252d9e0337446b88dc9d3c929683b62', 'request_id': 'req-2925250f-8eae-43da-905f-8d3e22de4831'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:04.290452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-2925250f-8eae-43da-905f-8d3e22de4831 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2252d9e0337446b88dc9d3c929683b62', 'request_id': 'req-2925250f-8eae-43da-905f-8d3e22de4831'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:04.324702 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-2925250f-8eae-43da-905f-8d3e22de4831 admin admin] The token cache is being invalidate because project 2252d9e0337446b88dc9d3c929683b62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:20:04.324702 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-2925250f-8eae-43da-905f-8d3e22de4831 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:04.324702 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-2925250f-8eae-43da-905f-8d3e22de4831 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:04.324702 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 977/1975] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:20:04 2022] DELETE /identity/v3/projects/2252d9e0337446b88dc9d3c929683b62 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:04.325060 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad91138-a570-44d0-9095-2e949668cc8c service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:04.336693 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad91138-a570-44d0-9095-2e949668cc8c service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:04.451549 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff75f8-a49c-4926-8d23-7cb6e9174cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:04.452275 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff75f8-a49c-4926-8d23-7cb6e9174cf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:04.452562 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff75f8-a49c-4926-8d23-7cb6e9174cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:04.452833 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff75f8-a49c-4926-8d23-7cb6e9174cf0 admin admin] PATH_INFO: `/v3/users/90732a9d0b034748879dbb1da4855d5b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:04.454542 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff75f8-a49c-4926-8d23-7cb6e9174cf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90732a9d0b034748879dbb1da4855d5b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:04.457362 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff75f8-a49c-4926-8d23-7cb6e9174cf0 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:04.535522 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-f5ff75f8-a49c-4926-8d23-7cb6e9174cf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90732a9d0b034748879dbb1da4855d5b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:04.544344 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-f5ff75f8-a49c-4926-8d23-7cb6e9174cf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90732a9d0b034748879dbb1da4855d5b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:04.570375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-f5ff75f8-a49c-4926-8d23-7cb6e9174cf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90732a9d0b034748879dbb1da4855d5b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:04.570999 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-f5ff75f8-a49c-4926-8d23-7cb6e9174cf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90732a9d0b034748879dbb1da4855d5b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:04.596451 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-f5ff75f8-a49c-4926-8d23-7cb6e9174cf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90732a9d0b034748879dbb1da4855d5b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:04.597129 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:04.605596 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 999/1976] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:20:04 2022] DELETE /identity/v3/users/90732a9d0b034748879dbb1da4855d5b => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:20:04.901423 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2925250f-8eae-43da-905f-8d3e22de4831 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:04.904587 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-2925250f-8eae-43da-905f-8d3e22de4831 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:04.926501 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2925250f-8eae-43da-905f-8d3e22de4831 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:04.983173 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff75f8-a49c-4926-8d23-7cb6e9174cf0 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:04.984670 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f5ff75f8-a49c-4926-8d23-7cb6e9174cf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:05.005010 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca2f2a6-2682-43b2-8677-d468e6d82edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:05.005989 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2f2a6-2682-43b2-8677-d468e6d82edb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:05.006465 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2f2a6-2682-43b2-8677-d468e6d82edb admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:05.006913 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2f2a6-2682-43b2-8677-d468e6d82edb admin admin] PATH_INFO: `/v3/projects/683a709a8e804ffe851db24f709b5b48` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:05.008104 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca2f2a6-2682-43b2-8677-d468e6d82edb admin admin] RBAC: Authorizing `identity:delete_project(project_id=683a709a8e804ffe851db24f709b5b48)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:05.010086 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca2f2a6-2682-43b2-8677-d468e6d82edb admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:05.065002 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0ca2f2a6-2682-43b2-8677-d468e6d82edb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '683a709a8e804ffe851db24f709b5b48', 'request_id': 'req-0ca2f2a6-2682-43b2-8677-d468e6d82edb'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:05.067348 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff75f8-a49c-4926-8d23-7cb6e9174cf0 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:05.075394 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0ca2f2a6-2682-43b2-8677-d468e6d82edb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '683a709a8e804ffe851db24f709b5b48', 'request_id': 'req-0ca2f2a6-2682-43b2-8677-d468e6d82edb'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:05.082196 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0ca2f2a6-2682-43b2-8677-d468e6d82edb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '683a709a8e804ffe851db24f709b5b48', 'request_id': 'req-0ca2f2a6-2682-43b2-8677-d468e6d82edb'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:05.101310 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0ca2f2a6-2682-43b2-8677-d468e6d82edb admin admin] The token cache is being invalidate because project 683a709a8e804ffe851db24f709b5b48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:20:05.101876 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0ca2f2a6-2682-43b2-8677-d468e6d82edb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:05.102746 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0ca2f2a6-2682-43b2-8677-d468e6d82edb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:05.104530 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 978/1977] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:20:04 2022] DELETE /identity/v3/projects/683a709a8e804ffe851db24f709b5b48 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:05.110402 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c4add-a45e-497a-b222-8a7709d94373 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:05.110684 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c4add-a45e-497a-b222-8a7709d94373 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:05.111122 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c4add-a45e-497a-b222-8a7709d94373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:05.136157 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-311c4add-a45e-497a-b222-8a7709d94373 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:05.137878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-311c4add-a45e-497a-b222-8a7709d94373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:05.156288 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141fb137-91bf-4384-97bc-f3716b0eab8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:05.157466 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141fb137-91bf-4384-97bc-f3716b0eab8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:05.157887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141fb137-91bf-4384-97bc-f3716b0eab8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:05.159053 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141fb137-91bf-4384-97bc-f3716b0eab8d admin admin] PATH_INFO: `/v3/users/3435046783404393948a40d2d61043a3` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:05.161194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141fb137-91bf-4384-97bc-f3716b0eab8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3435046783404393948a40d2d61043a3)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:05.164294 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141fb137-91bf-4384-97bc-f3716b0eab8d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:05.199826 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:05.214163 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 979/1978] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:20:05.220699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3eeaa9-8bf6-4faf-942b-15c90f04ed9b None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:05.220982 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3eeaa9-8bf6-4faf-942b-15c90f04ed9b None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:05.221248 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3eeaa9-8bf6-4faf-942b-15c90f04ed9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:05.232664 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-141fb137-91bf-4384-97bc-f3716b0eab8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3435046783404393948a40d2d61043a3'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:05.242069 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-1c3eeaa9-8bf6-4faf-942b-15c90f04ed9b None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:05.243135 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-1c3eeaa9-8bf6-4faf-942b-15c90f04ed9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:05.248345 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-141fb137-91bf-4384-97bc-f3716b0eab8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3435046783404393948a40d2d61043a3'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:05.254353 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 980/1979] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:05.256787 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3eeaa9-8bf6-4faf-942b-15c90f04ed9b None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:05.259855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-141fb137-91bf-4384-97bc-f3716b0eab8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3435046783404393948a40d2d61043a3'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:05.260617 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-141fb137-91bf-4384-97bc-f3716b0eab8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3435046783404393948a40d2d61043a3'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:05.268160 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3eeaa9-8bf6-4faf-942b-15c90f04ed9b None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:05.273224 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79aed7e-ad85-41ae-b4d7-1addbf0dca05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:05.274061 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79aed7e-ad85-41ae-b4d7-1addbf0dca05 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:05.274408 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79aed7e-ad85-41ae-b4d7-1addbf0dca05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:05.274751 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79aed7e-ad85-41ae-b4d7-1addbf0dca05 admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:05.276416 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79aed7e-ad85-41ae-b4d7-1addbf0dca05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:20:05.276990 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79aed7e-ad85-41ae-b4d7-1addbf0dca05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:05.279181 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79aed7e-ad85-41ae-b4d7-1addbf0dca05 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:05.281967 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-141fb137-91bf-4384-97bc-f3716b0eab8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3435046783404393948a40d2d61043a3'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:05.298401 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1000/1980] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:20:04 2022] DELETE /identity/v3/users/3435046783404393948a40d2d61043a3 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:05.300040 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 981/1981] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:20:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:20:05.300634 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141fb137-91bf-4384-97bc-f3716b0eab8d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:05.302267 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-141fb137-91bf-4384-97bc-f3716b0eab8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:05.302892 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c358d-0bcf-4573-a145-505ea0fbb179 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:05.304107 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c358d-0bcf-4573-a145-505ea0fbb179 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:05.304614 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c358d-0bcf-4573-a145-505ea0fbb179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:05.342480 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-270c358d-0bcf-4573-a145-505ea0fbb179 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:05.346165 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-270c358d-0bcf-4573-a145-505ea0fbb179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:05.366496 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:05.468586 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141fb137-91bf-4384-97bc-f3716b0eab8d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:05.474923 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5696a92d-c1ed-4851-8a6c-cfc666c89041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:05.475893 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696a92d-c1ed-4851-8a6c-cfc666c89041 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:05.476485 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696a92d-c1ed-4851-8a6c-cfc666c89041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:05.477063 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696a92d-c1ed-4851-8a6c-cfc666c89041 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:05.477872 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5696a92d-c1ed-4851-8a6c-cfc666c89041 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:05.480057 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5696a92d-c1ed-4851-8a6c-cfc666c89041 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:05.527471 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1001/1982] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:20:05 2022] POST /identity/v3/projects => generated 391 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:20:05.531938 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5696a92d-c1ed-4851-8a6c-cfc666c89041 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:05.538677 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 982/1983] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:05.593677 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c358d-0bcf-4573-a145-505ea0fbb179 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:05.595943 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-270c358d-0bcf-4573-a145-505ea0fbb179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:05.692272 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5696a92d-c1ed-4851-8a6c-cfc666c89041 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:05.713676 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0626e1e-f57a-4d35-8cee-9966628fe1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:05.715139 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0626e1e-f57a-4d35-8cee-9966628fe1ed admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:05.715429 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0626e1e-f57a-4d35-8cee-9966628fe1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:05.717844 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0626e1e-f57a-4d35-8cee-9966628fe1ed admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:05.720471 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0626e1e-f57a-4d35-8cee-9966628fe1ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:05.723838 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0626e1e-f57a-4d35-8cee-9966628fe1ed admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:05.746331 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c358d-0bcf-4573-a145-505ea0fbb179 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:05.786653 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1002/1984] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:20:05 2022] POST /identity/v3/users => generated 296 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:20:05.793654 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0626e1e-f57a-4d35-8cee-9966628fe1ed admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:05.816600 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0626e1e-f57a-4d35-8cee-9966628fe1ed admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:05.826804 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f1d9a-f6ff-485f-95f4-130ea0e8b474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:05.827671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f1d9a-f6ff-485f-95f4-130ea0e8b474 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:05.828397 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f1d9a-f6ff-485f-95f4-130ea0e8b474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:05.828929 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f1d9a-f6ff-485f-95f4-130ea0e8b474 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:05.830074 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9f1d9a-f6ff-485f-95f4-130ea0e8b474 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:05.833891 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9f1d9a-f6ff-485f-95f4-130ea0e8b474 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:05.841763 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed7d13b-38b4-414d-86a8-33c2fc69e7ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:05.843460 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7d13b-38b4-414d-86a8-33c2fc69e7ec service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:05.843909 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7d13b-38b4-414d-86a8-33c2fc69e7ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:05.844368 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7d13b-38b4-414d-86a8-33c2fc69e7ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:05.858168 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1003/1985] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:05 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:20:05.863317 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f1d9a-f6ff-485f-95f4-130ea0e8b474 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:05.896838 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f1d9a-f6ff-485f-95f4-130ea0e8b474 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:05.902881 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3879e9f2-c22d-4961-86a7-3598d4ed4c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:05.903843 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3879e9f2-c22d-4961-86a7-3598d4ed4c6a admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:05.904323 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3879e9f2-c22d-4961-86a7-3598d4ed4c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:05.904804 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3879e9f2-c22d-4961-86a7-3598d4ed4c6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:05.905547 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3879e9f2-c22d-4961-86a7-3598d4ed4c6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:05.907289 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3879e9f2-c22d-4961-86a7-3598d4ed4c6a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:05.910524 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed7d13b-38b4-414d-86a8-33c2fc69e7ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:05.912984 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed7d13b-38b4-414d-86a8-33c2fc69e7ec service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:05.923290 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1004/1986] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:05 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:20:05.928711 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3879e9f2-c22d-4961-86a7-3598d4ed4c6a admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:05.948005 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3879e9f2-c22d-4961-86a7-3598d4ed4c6a admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:05.952382 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 983/1987] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:20:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:20:05.959810 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e69549b-67d1-4b9f-bbef-5e27702b8389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:05.960996 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e69549b-67d1-4b9f-bbef-5e27702b8389 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:05.961544 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e69549b-67d1-4b9f-bbef-5e27702b8389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:05.962124 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e69549b-67d1-4b9f-bbef-5e27702b8389 admin admin] PATH_INFO: `/v3/projects/e9ced8830de7450dbf632972cc4cd224/users/22b48e6baa1a4c3da5cbcf38e90a5d7d/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:05.999263 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e69549b-67d1-4b9f-bbef-5e27702b8389 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9ced8830de7450dbf632972cc4cd224, user_id=22b48e6baa1a4c3da5cbcf38e90a5d7d, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:06.002414 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e69549b-67d1-4b9f-bbef-5e27702b8389 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:06.028927 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1005/1988] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:20:05 2022] PUT /identity/v3/projects/e9ced8830de7450dbf632972cc4cd224/users/22b48e6baa1a4c3da5cbcf38e90a5d7d/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:06.055121 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f6c34-4ba3-4c08-ae5d-923a825006d7 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:06.055594 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f6c34-4ba3-4c08-ae5d-923a825006d7 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:06.056056 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f6c34-4ba3-4c08-ae5d-923a825006d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:06.128194 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-797f6c34-4ba3-4c08-ae5d-923a825006d7 None None] MFA Rules not processed for user `22b48e6baa1a4c3da5cbcf38e90a5d7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:06.146371 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-797f6c34-4ba3-4c08-ae5d-923a825006d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:06.213731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:06.309464 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 984/1989] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:06 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:06.315102 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e69549b-67d1-4b9f-bbef-5e27702b8389 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:06.359458 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797f6c34-4ba3-4c08-ae5d-923a825006d7 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:06.361891 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-797f6c34-4ba3-4c08-ae5d-923a825006d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:06.383992 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:06.406239 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e69549b-67d1-4b9f-bbef-5e27702b8389 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:06.423095 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c200d95f-5274-418d-9912-5b278fc77faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:06.423095 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200d95f-5274-418d-9912-5b278fc77faf admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:06.423095 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200d95f-5274-418d-9912-5b278fc77faf admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:06.423095 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200d95f-5274-418d-9912-5b278fc77faf admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:06.424180 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c200d95f-5274-418d-9912-5b278fc77faf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:06.424180 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c200d95f-5274-418d-9912-5b278fc77faf admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:06.460014 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1006/1990] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:20:06 2022] POST /identity/v3/projects => generated 393 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:20:06.462493 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c200d95f-5274-418d-9912-5b278fc77faf admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:06.546699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797f6c34-4ba3-4c08-ae5d-923a825006d7 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:06.556833 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c200d95f-5274-418d-9912-5b278fc77faf admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:06.561573 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d57a48-dcaf-42db-adf7-5a577b343639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:06.562234 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d57a48-dcaf-42db-adf7-5a577b343639 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:06.562512 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d57a48-dcaf-42db-adf7-5a577b343639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:06.562847 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d57a48-dcaf-42db-adf7-5a577b343639 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:06.563398 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d57a48-dcaf-42db-adf7-5a577b343639 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:06.565357 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d57a48-dcaf-42db-adf7-5a577b343639 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:06.569303 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df2a38-3ed3-47af-9f9c-af97687087ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:06.569968 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df2a38-3ed3-47af-9f9c-af97687087ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:06.570275 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df2a38-3ed3-47af-9f9c-af97687087ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:06.570724 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df2a38-3ed3-47af-9f9c-af97687087ba admin admin] PATH_INFO: `/v3/users/5ed9b41695f746868768127dad722f7e` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:06.571877 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85df2a38-3ed3-47af-9f9c-af97687087ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ed9b41695f746868768127dad722f7e)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:06.573822 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85df2a38-3ed3-47af-9f9c-af97687087ba admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:06.589617 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1007/1991] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:20:06 2022] POST /identity/v3/users => generated 303 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:20:06.590747 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d57a48-dcaf-42db-adf7-5a577b343639 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:06.592612 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c2d57a48-dcaf-42db-adf7-5a577b343639 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:06.604465 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d57a48-dcaf-42db-adf7-5a577b343639 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:06.609750 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5632a4-b2b2-4594-a03c-c66db36be6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:06.610799 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5632a4-b2b2-4594-a03c-c66db36be6db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:06.611146 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5632a4-b2b2-4594-a03c-c66db36be6db admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:06.611492 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5632a4-b2b2-4594-a03c-c66db36be6db admin admin] PATH_INFO: `/v3/projects/a12bfd6a0f9d4b559a0c2f5f26af4933` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:06.612755 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5632a4-b2b2-4594-a03c-c66db36be6db admin admin] RBAC: Authorizing `identity:delete_project(project_id=a12bfd6a0f9d4b559a0c2f5f26af4933)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:06.614854 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5632a4-b2b2-4594-a03c-c66db36be6db admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:06.629750 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-85df2a38-3ed3-47af-9f9c-af97687087ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ed9b41695f746868768127dad722f7e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:06.636715 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-85df2a38-3ed3-47af-9f9c-af97687087ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ed9b41695f746868768127dad722f7e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:06.637375 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-85df2a38-3ed3-47af-9f9c-af97687087ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ed9b41695f746868768127dad722f7e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:06.650351 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-85df2a38-3ed3-47af-9f9c-af97687087ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ed9b41695f746868768127dad722f7e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:06.651683 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-85df2a38-3ed3-47af-9f9c-af97687087ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ed9b41695f746868768127dad722f7e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:06.666625 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dd5632a4-b2b2-4594-a03c-c66db36be6db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a12bfd6a0f9d4b559a0c2f5f26af4933', 'request_id': 'req-dd5632a4-b2b2-4594-a03c-c66db36be6db'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:06.672183 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 985/1992] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:20:06 2022] DELETE /identity/v3/users/5ed9b41695f746868768127dad722f7e => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:20:06.673051 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df2a38-3ed3-47af-9f9c-af97687087ba admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:06.673523 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dd5632a4-b2b2-4594-a03c-c66db36be6db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a12bfd6a0f9d4b559a0c2f5f26af4933', 'request_id': 'req-dd5632a4-b2b2-4594-a03c-c66db36be6db'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:06.674428 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-85df2a38-3ed3-47af-9f9c-af97687087ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:06.682800 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dd5632a4-b2b2-4594-a03c-c66db36be6db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a12bfd6a0f9d4b559a0c2f5f26af4933', 'request_id': 'req-dd5632a4-b2b2-4594-a03c-c66db36be6db'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:06.693372 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dd5632a4-b2b2-4594-a03c-c66db36be6db admin admin] The token cache is being invalidate because project a12bfd6a0f9d4b559a0c2f5f26af4933 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:20:06.693655 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dd5632a4-b2b2-4594-a03c-c66db36be6db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:06.694327 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dd5632a4-b2b2-4594-a03c-c66db36be6db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:06.695854 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1008/1993] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:20:06 2022] DELETE /identity/v3/projects/a12bfd6a0f9d4b559a0c2f5f26af4933 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:06.697038 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3db1d5-8bb6-4dbe-a6db-ac6f877bd405 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:06.697382 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3db1d5-8bb6-4dbe-a6db-ac6f877bd405 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:06.697653 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3db1d5-8bb6-4dbe-a6db-ac6f877bd405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:06.717396 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ab3db1d5-8bb6-4dbe-a6db-ac6f877bd405 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:06.718702 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ab3db1d5-8bb6-4dbe-a6db-ac6f877bd405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:06.731300 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:06.751087 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df2a38-3ed3-47af-9f9c-af97687087ba admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:06.756430 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1432352f-ca60-4619-88fa-8e92902a1fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:06.757365 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432352f-ca60-4619-88fa-8e92902a1fe7 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:06.757365 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432352f-ca60-4619-88fa-8e92902a1fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:06.757679 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432352f-ca60-4619-88fa-8e92902a1fe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:06.758522 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1432352f-ca60-4619-88fa-8e92902a1fe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:06.760274 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1432352f-ca60-4619-88fa-8e92902a1fe7 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:06.766731 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:06.771361 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 986/1994] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:06 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:20:06.772358 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1432352f-ca60-4619-88fa-8e92902a1fe7 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:06.774172 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-1432352f-ca60-4619-88fa-8e92902a1fe7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:06.789357 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1009/1995] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:06.790359 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3db1d5-8bb6-4dbe-a6db-ac6f877bd405 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:06.791728 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1432352f-ca60-4619-88fa-8e92902a1fe7 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:06.799621 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ab3db1d5-8bb6-4dbe-a6db-ac6f877bd405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:06.805577 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b866be-0d8d-4e66-881f-05ade525d846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:06.806387 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b866be-0d8d-4e66-881f-05ade525d846 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:06.806788 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b866be-0d8d-4e66-881f-05ade525d846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:06.807060 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b866be-0d8d-4e66-881f-05ade525d846 admin admin] PATH_INFO: `/v3/users/e5322876703b4d73b7cae1e7bf39b1b4` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:06.808535 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b866be-0d8d-4e66-881f-05ade525d846 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5322876703b4d73b7cae1e7bf39b1b4)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:06.810586 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b866be-0d8d-4e66-881f-05ade525d846 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:06.812110 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3db1d5-8bb6-4dbe-a6db-ac6f877bd405 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:06.818574 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed21aa92-34fd-40bb-bbf9-b92a726b3bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:06.819247 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed21aa92-34fd-40bb-bbf9-b92a726b3bcc admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:06.819532 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed21aa92-34fd-40bb-bbf9-b92a726b3bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:06.819805 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed21aa92-34fd-40bb-bbf9-b92a726b3bcc admin admin] PATH_INFO: `/v3/projects/16c9d6e4f76849d1b6b7512b8b54df97/users/b33d190ced304b16a2735464ad50d12f/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:06.840554 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed21aa92-34fd-40bb-bbf9-b92a726b3bcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=16c9d6e4f76849d1b6b7512b8b54df97, user_id=b33d190ced304b16a2735464ad50d12f, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:06.843552 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed21aa92-34fd-40bb-bbf9-b92a726b3bcc admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:06.865573 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-23b866be-0d8d-4e66-881f-05ade525d846 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5322876703b4d73b7cae1e7bf39b1b4'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:06.869056 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1010/1996] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:20:06 2022] PUT /identity/v3/projects/16c9d6e4f76849d1b6b7512b8b54df97/users/b33d190ced304b16a2735464ad50d12f/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:06.869980 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed21aa92-34fd-40bb-bbf9-b92a726b3bcc admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:06.871788 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ed21aa92-34fd-40bb-bbf9-b92a726b3bcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:06.874419 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-23b866be-0d8d-4e66-881f-05ade525d846 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5322876703b4d73b7cae1e7bf39b1b4'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:06.876036 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-23b866be-0d8d-4e66-881f-05ade525d846 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5322876703b4d73b7cae1e7bf39b1b4'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:06.884470 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-23b866be-0d8d-4e66-881f-05ade525d846 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5322876703b4d73b7cae1e7bf39b1b4'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:06.885176 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-23b866be-0d8d-4e66-881f-05ade525d846 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5322876703b4d73b7cae1e7bf39b1b4'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:06.905936 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 987/1997] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:20:06 2022] DELETE /identity/v3/users/e5322876703b4d73b7cae1e7bf39b1b4 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:06.908543 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b866be-0d8d-4e66-881f-05ade525d846 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:06.908543 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-23b866be-0d8d-4e66-881f-05ade525d846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:06.947126 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed21aa92-34fd-40bb-bbf9-b92a726b3bcc admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:06.999694 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b866be-0d8d-4e66-881f-05ade525d846 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:07.007056 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843e88bf-6f5c-47c2-ad36-39934ec9e114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:07.007927 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e88bf-6f5c-47c2-ad36-39934ec9e114 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:07.008214 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e88bf-6f5c-47c2-ad36-39934ec9e114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:07.008489 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e88bf-6f5c-47c2-ad36-39934ec9e114 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:07.009273 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843e88bf-6f5c-47c2-ad36-39934ec9e114 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:07.011079 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843e88bf-6f5c-47c2-ad36-39934ec9e114 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:07.018740 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494848b-1539-4c40-9c15-9f911f347599 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:07.019645 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494848b-1539-4c40-9c15-9f911f347599 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:07.020104 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494848b-1539-4c40-9c15-9f911f347599 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:07.020654 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494848b-1539-4c40-9c15-9f911f347599 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:07.023429 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3494848b-1539-4c40-9c15-9f911f347599 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:07.027510 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 988/1998] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:06 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:20:07.032112 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843e88bf-6f5c-47c2-ad36-39934ec9e114 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:07.038675 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3494848b-1539-4c40-9c15-9f911f347599 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:07.040733 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3494848b-1539-4c40-9c15-9f911f347599 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:07.043072 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843e88bf-6f5c-47c2-ad36-39934ec9e114 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:07.049159 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c46e25-4945-4962-a469-f1f5a51e6c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:07.049822 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c46e25-4945-4962-a469-f1f5a51e6c7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:07.050133 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c46e25-4945-4962-a469-f1f5a51e6c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:07.050456 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c46e25-4945-4962-a469-f1f5a51e6c7a admin admin] PATH_INFO: `/v3/projects/16c9d6e4f76849d1b6b7512b8b54df97/users/b33d190ced304b16a2735464ad50d12f/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:07.052473 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c46e25-4945-4962-a469-f1f5a51e6c7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=16c9d6e4f76849d1b6b7512b8b54df97, user_id=b33d190ced304b16a2735464ad50d12f, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:07.052583 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1011/1999] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:20:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:20:07.055268 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c46e25-4945-4962-a469-f1f5a51e6c7a admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:07.067224 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 989/2000] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:20:07 2022] PUT /identity/v3/projects/16c9d6e4f76849d1b6b7512b8b54df97/users/b33d190ced304b16a2735464ad50d12f/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:07.078107 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f245d-3207-441b-beb2-4bd6c1a3640a None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:07.078415 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f245d-3207-441b-beb2-4bd6c1a3640a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:07.078706 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f245d-3207-441b-beb2-4bd6c1a3640a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:07.116985 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a96f245d-3207-441b-beb2-4bd6c1a3640a None None] MFA Rules not processed for user `b33d190ced304b16a2735464ad50d12f`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:07.118139 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a96f245d-3207-441b-beb2-4bd6c1a3640a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:07.240584 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1012/2001] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:07 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:20:07.253331 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c46e25-4945-4962-a469-f1f5a51e6c7a admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:07.255624 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-05c46e25-4945-4962-a469-f1f5a51e6c7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:07.301324 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f245d-3207-441b-beb2-4bd6c1a3640a None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:07.303307 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a96f245d-3207-441b-beb2-4bd6c1a3640a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:07.324199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:07.324423 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:07.349767 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c46e25-4945-4962-a469-f1f5a51e6c7a admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:07.355144 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dce76e-b392-4d03-8701-4e33fa99aeac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:07.355866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dce76e-b392-4d03-8701-4e33fa99aeac service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:07.356210 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dce76e-b392-4d03-8701-4e33fa99aeac service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:07.357540 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dce76e-b392-4d03-8701-4e33fa99aeac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:07.369544 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dce76e-b392-4d03-8701-4e33fa99aeac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:07.371441 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dce76e-b392-4d03-8701-4e33fa99aeac service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:07.384822 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 990/2002] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:20:07 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:20:07.401835 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f245d-3207-441b-beb2-4bd6c1a3640a None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:07.406777 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f57c5d-1b3b-41e9-9110-46afeab68c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:07.407512 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f57c5d-1b3b-41e9-9110-46afeab68c99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:07.407875 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f57c5d-1b3b-41e9-9110-46afeab68c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:07.408212 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f57c5d-1b3b-41e9-9110-46afeab68c99 admin admin] PATH_INFO: `/v3/projects/d733981362f94187826314d8d9829774` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:07.409542 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f57c5d-1b3b-41e9-9110-46afeab68c99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d733981362f94187826314d8d9829774)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:07.411642 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f57c5d-1b3b-41e9-9110-46afeab68c99 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:07.412104 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dce76e-b392-4d03-8701-4e33fa99aeac service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:07.415249 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-23dce76e-b392-4d03-8701-4e33fa99aeac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:07.445676 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dce76e-b392-4d03-8701-4e33fa99aeac service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:07.451750 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34afbedd-19fa-466f-a003-9a76fb43135f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:07.452561 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34afbedd-19fa-466f-a003-9a76fb43135f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:07.452818 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34afbedd-19fa-466f-a003-9a76fb43135f admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:07.453080 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34afbedd-19fa-466f-a003-9a76fb43135f admin admin] PATH_INFO: `/v3/projects/99ca2cb2b3c44c2a8bfec89fe42b8387` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:07.454731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34afbedd-19fa-466f-a003-9a76fb43135f admin admin] RBAC: Authorizing `identity:delete_project(project_id=99ca2cb2b3c44c2a8bfec89fe42b8387)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:07.456804 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34afbedd-19fa-466f-a003-9a76fb43135f admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:07.468777 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-75f57c5d-1b3b-41e9-9110-46afeab68c99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd733981362f94187826314d8d9829774', 'request_id': 'req-75f57c5d-1b3b-41e9-9110-46afeab68c99'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:07.475394 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-75f57c5d-1b3b-41e9-9110-46afeab68c99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd733981362f94187826314d8d9829774', 'request_id': 'req-75f57c5d-1b3b-41e9-9110-46afeab68c99'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:07.483576 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-75f57c5d-1b3b-41e9-9110-46afeab68c99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd733981362f94187826314d8d9829774', 'request_id': 'req-75f57c5d-1b3b-41e9-9110-46afeab68c99'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:07.496131 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-75f57c5d-1b3b-41e9-9110-46afeab68c99 admin admin] The token cache is being invalidate because project d733981362f94187826314d8d9829774 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:20:07.496420 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-75f57c5d-1b3b-41e9-9110-46afeab68c99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:07.497583 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-75f57c5d-1b3b-41e9-9110-46afeab68c99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:07.499134 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1013/2003] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:20:07 2022] DELETE /identity/v3/projects/d733981362f94187826314d8d9829774 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:20:07.502908 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f57c5d-1b3b-41e9-9110-46afeab68c99 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:07.504300 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-75f57c5d-1b3b-41e9-9110-46afeab68c99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:07.515381 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-34afbedd-19fa-466f-a003-9a76fb43135f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99ca2cb2b3c44c2a8bfec89fe42b8387', 'request_id': 'req-34afbedd-19fa-466f-a003-9a76fb43135f'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:07.522953 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-34afbedd-19fa-466f-a003-9a76fb43135f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99ca2cb2b3c44c2a8bfec89fe42b8387', 'request_id': 'req-34afbedd-19fa-466f-a003-9a76fb43135f'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:07.529002 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-34afbedd-19fa-466f-a003-9a76fb43135f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99ca2cb2b3c44c2a8bfec89fe42b8387', 'request_id': 'req-34afbedd-19fa-466f-a003-9a76fb43135f'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:07.544141 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-34afbedd-19fa-466f-a003-9a76fb43135f admin admin] The token cache is being invalidate because project 99ca2cb2b3c44c2a8bfec89fe42b8387 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:20:07.544517 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-34afbedd-19fa-466f-a003-9a76fb43135f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:07.545244 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-34afbedd-19fa-466f-a003-9a76fb43135f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:07.547083 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 991/2004] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:20:07 2022] DELETE /identity/v3/projects/99ca2cb2b3c44c2a8bfec89fe42b8387 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:20:07.548106 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454c0f4-1444-4153-9ce4-cb1293ccc4e4 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:07.548456 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454c0f4-1444-4153-9ce4-cb1293ccc4e4 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:07.548792 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454c0f4-1444-4153-9ce4-cb1293ccc4e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:07.568848 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-6454c0f4-1444-4153-9ce4-cb1293ccc4e4 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:07.570419 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-6454c0f4-1444-4153-9ce4-cb1293ccc4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:07.579946 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f57c5d-1b3b-41e9-9110-46afeab68c99 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:07.646831 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c808c6-4fd9-4fea-b9c9-fcf46b721069 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:07.647380 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c808c6-4fd9-4fea-b9c9-fcf46b721069 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:07.647658 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c808c6-4fd9-4fea-b9c9-fcf46b721069 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:07.647918 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c808c6-4fd9-4fea-b9c9-fcf46b721069 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:07.649718 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f8c808c6-4fd9-4fea-b9c9-fcf46b721069 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:07.664336 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:07.667349 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 992/2005] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:07.668251 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454c0f4-1444-4153-9ce4-cb1293ccc4e4 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:07.669686 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-6454c0f4-1444-4153-9ce4-cb1293ccc4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:07.684003 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454c0f4-1444-4153-9ce4-cb1293ccc4e4 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:07.689291 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0bd3c0-6c43-4e53-a260-e324e40a9646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:07.690135 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0bd3c0-6c43-4e53-a260-e324e40a9646 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:07.690414 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0bd3c0-6c43-4e53-a260-e324e40a9646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:07.690677 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0bd3c0-6c43-4e53-a260-e324e40a9646 admin admin] PATH_INFO: `/v3/users/e7b35249c1544746bd8d4340821cf30b` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:07.691957 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0bd3c0-6c43-4e53-a260-e324e40a9646 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7b35249c1544746bd8d4340821cf30b)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:07.693965 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0bd3c0-6c43-4e53-a260-e324e40a9646 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:07.742759 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c808c6-4fd9-4fea-b9c9-fcf46b721069 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:07.744316 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c808c6-4fd9-4fea-b9c9-fcf46b721069 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:07.753584 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1014/2006] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:20:07 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:20:07.756650 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16e8b5-eca8-49c8-8112-e91595ea90d1 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:07.756927 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16e8b5-eca8-49c8-8112-e91595ea90d1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:07.757202 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16e8b5-eca8-49c8-8112-e91595ea90d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:07.759411 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-fb0bd3c0-6c43-4e53-a260-e324e40a9646 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7b35249c1544746bd8d4340821cf30b'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:07.770615 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-fb0bd3c0-6c43-4e53-a260-e324e40a9646 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7b35249c1544746bd8d4340821cf30b'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:07.772042 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-fb0bd3c0-6c43-4e53-a260-e324e40a9646 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7b35249c1544746bd8d4340821cf30b'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:07.780671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-5c16e8b5-eca8-49c8-8112-e91595ea90d1 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:07.782977 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5c16e8b5-eca8-49c8-8112-e91595ea90d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:07.783675 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:07.789580 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-fb0bd3c0-6c43-4e53-a260-e324e40a9646 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7b35249c1544746bd8d4340821cf30b'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:07.792331 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-fb0bd3c0-6c43-4e53-a260-e324e40a9646 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7b35249c1544746bd8d4340821cf30b'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:07.798597 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1015/2007] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:07.804093 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c16e8b5-eca8-49c8-8112-e91595ea90d1 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:07.819648 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c16e8b5-eca8-49c8-8112-e91595ea90d1 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:07.826122 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bc8f16-8e9c-4696-89e8-edab17109e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:07.826659 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 993/2008] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:20:07 2022] DELETE /identity/v3/users/e7b35249c1544746bd8d4340821cf30b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:07.826979 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc8f16-8e9c-4696-89e8-edab17109e4b admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:07.827457 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc8f16-8e9c-4696-89e8-edab17109e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:07.828599 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc8f16-8e9c-4696-89e8-edab17109e4b admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:07.830167 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bc8f16-8e9c-4696-89e8-edab17109e4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:20:07.830860 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bc8f16-8e9c-4696-89e8-edab17109e4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:07.832530 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bc8f16-8e9c-4696-89e8-edab17109e4b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:07.849195 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1016/2009] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:20:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:20:07.854438 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0bd3c0-6c43-4e53-a260-e324e40a9646 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:07.949433 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0bd3c0-6c43-4e53-a260-e324e40a9646 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:07.955117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00efb4c-b5ec-41f0-ab3d-c56208fb8875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:07.955730 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00efb4c-b5ec-41f0-ab3d-c56208fb8875 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:07.956009 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00efb4c-b5ec-41f0-ab3d-c56208fb8875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:07.956273 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00efb4c-b5ec-41f0-ab3d-c56208fb8875 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:07.956967 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00efb4c-b5ec-41f0-ab3d-c56208fb8875 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:07.958937 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00efb4c-b5ec-41f0-ab3d-c56208fb8875 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:08.021128 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 994/2010] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:20:07 2022] POST /identity/v3/projects => generated 407 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:20:08.025242 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bc8f16-8e9c-4696-89e8-edab17109e4b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:08.073768 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bc8f16-8e9c-4696-89e8-edab17109e4b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:08.194719 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf4afc-36b8-48d1-81d7-79368bb12a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:08.194719 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf4afc-36b8-48d1-81d7-79368bb12a49 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:08.194719 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf4afc-36b8-48d1-81d7-79368bb12a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:08.194719 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf4afc-36b8-48d1-81d7-79368bb12a49 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:08.195084 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bf4afc-36b8-48d1-81d7-79368bb12a49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:08.195261 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bf4afc-36b8-48d1-81d7-79368bb12a49 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:08.247012 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1017/2011] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:20:08 2022] POST /identity/v3/users => generated 304 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:20:08.255256 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00efb4c-b5ec-41f0-ab3d-c56208fb8875 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:08.273688 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00efb4c-b5ec-41f0-ab3d-c56208fb8875 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:08.279187 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd7ebf1-cca7-4d8b-8a08-a6a9f32da3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:08.279934 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7ebf1-cca7-4d8b-8a08-a6a9f32da3c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:08.280204 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7ebf1-cca7-4d8b-8a08-a6a9f32da3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:08.280477 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7ebf1-cca7-4d8b-8a08-a6a9f32da3c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:08.281517 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd7ebf1-cca7-4d8b-8a08-a6a9f32da3c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:08.283188 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd7ebf1-cca7-4d8b-8a08-a6a9f32da3c3 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:08.298858 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 995/2012] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:08 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:20:08.302855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf4afc-36b8-48d1-81d7-79368bb12a49 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:08.314996 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf4afc-36b8-48d1-81d7-79368bb12a49 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:08.336230 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb9d98c-aad9-4004-ba76-ce3023ec3070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:08.337167 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9d98c-aad9-4004-ba76-ce3023ec3070 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:08.337681 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9d98c-aad9-4004-ba76-ce3023ec3070 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:08.338322 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9d98c-aad9-4004-ba76-ce3023ec3070 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:08.339130 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb9d98c-aad9-4004-ba76-ce3023ec3070 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:08.340920 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb9d98c-aad9-4004-ba76-ce3023ec3070 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:08.357854 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1018/2013] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:08 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 05:20:08.359121 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd7ebf1-cca7-4d8b-8a08-a6a9f32da3c3 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:08.367337 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb9d98c-aad9-4004-ba76-ce3023ec3070 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:08.370326 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-eeb9d98c-aad9-4004-ba76-ce3023ec3070 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:08.370648 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd7ebf1-cca7-4d8b-8a08-a6a9f32da3c3 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:08.375394 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7bd054-2f9c-4c8a-ab5c-b9e1290268a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:08.376055 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7bd054-2f9c-4c8a-ab5c-b9e1290268a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:08.376333 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7bd054-2f9c-4c8a-ab5c-b9e1290268a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:08.376602 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7bd054-2f9c-4c8a-ab5c-b9e1290268a6 admin admin] PATH_INFO: `/v3/projects/046429fd547f41ae90b5354dfd8b075b/users/3bb39ec582b9410eab580e4e07871f07/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:08.393512 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7bd054-2f9c-4c8a-ab5c-b9e1290268a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=046429fd547f41ae90b5354dfd8b075b, user_id=3bb39ec582b9410eab580e4e07871f07, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:08.396390 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7bd054-2f9c-4c8a-ab5c-b9e1290268a6 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:08.411980 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb9d98c-aad9-4004-ba76-ce3023ec3070 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:08.412454 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 996/2014] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:20:08 2022] PUT /identity/v3/projects/046429fd547f41ae90b5354dfd8b075b/users/3bb39ec582b9410eab580e4e07871f07/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:08.425536 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac3257-bfcc-462f-9fe3-a09778ee155e None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:08.425867 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac3257-bfcc-462f-9fe3-a09778ee155e None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:08.426168 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac3257-bfcc-462f-9fe3-a09778ee155e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:08.459918 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-7dac3257-bfcc-462f-9fe3-a09778ee155e None None] MFA Rules not processed for user `3bb39ec582b9410eab580e4e07871f07`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:08.461180 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-7dac3257-bfcc-462f-9fe3-a09778ee155e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:08.488616 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ff6db1-ffc4-46cb-ba76-29dcd063ae50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:08.489250 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ff6db1-ffc4-46cb-ba76-29dcd063ae50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:08.489525 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ff6db1-ffc4-46cb-ba76-29dcd063ae50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:08.489792 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ff6db1-ffc4-46cb-ba76-29dcd063ae50 admin admin] PATH_INFO: `/v3/projects/382300da8cf0482a96ab096871b75675` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:08.491017 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ff6db1-ffc4-46cb-ba76-29dcd063ae50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=382300da8cf0482a96ab096871b75675)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:08.492884 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ff6db1-ffc4-46cb-ba76-29dcd063ae50 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:08.506343 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:08.541521 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 997/2015] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:08 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:08.546969 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac3257-bfcc-462f-9fe3-a09778ee155e None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:08.563966 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac3257-bfcc-462f-9fe3-a09778ee155e None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:08.565063 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-92ff6db1-ffc4-46cb-ba76-29dcd063ae50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '382300da8cf0482a96ab096871b75675', 'request_id': 'req-92ff6db1-ffc4-46cb-ba76-29dcd063ae50'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:08.574821 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-92ff6db1-ffc4-46cb-ba76-29dcd063ae50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '382300da8cf0482a96ab096871b75675', 'request_id': 'req-92ff6db1-ffc4-46cb-ba76-29dcd063ae50'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:08.584129 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:08.584675 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-92ff6db1-ffc4-46cb-ba76-29dcd063ae50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '382300da8cf0482a96ab096871b75675', 'request_id': 'req-92ff6db1-ffc4-46cb-ba76-29dcd063ae50'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:08.601095 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-92ff6db1-ffc4-46cb-ba76-29dcd063ae50 admin admin] The token cache is being invalidate because project 382300da8cf0482a96ab096871b75675 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:20:08.601391 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-92ff6db1-ffc4-46cb-ba76-29dcd063ae50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:08.602169 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-92ff6db1-ffc4-46cb-ba76-29dcd063ae50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:08.604202 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1019/2016] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:20:08 2022] DELETE /identity/v3/projects/382300da8cf0482a96ab096871b75675 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:08.609673 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f8a27-c3ed-4139-af1d-f3579f3996fc None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:08.610381 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f8a27-c3ed-4139-af1d-f3579f3996fc None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:08.610724 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f8a27-c3ed-4139-af1d-f3579f3996fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:08.636394 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-305f8a27-c3ed-4139-af1d-f3579f3996fc None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:08.637268 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-305f8a27-c3ed-4139-af1d-f3579f3996fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:08.642910 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f2d26d-fea3-48d6-b877-8922f3881713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:08.643468 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f2d26d-fea3-48d6-b877-8922f3881713 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:08.643789 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f2d26d-fea3-48d6-b877-8922f3881713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:08.644061 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f2d26d-fea3-48d6-b877-8922f3881713 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:08.649390 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f2d26d-fea3-48d6-b877-8922f3881713 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:08.649390 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f2d26d-fea3-48d6-b877-8922f3881713 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:08.665746 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1020/2017] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:08.668964 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a2b4d-7524-4b29-85a7-38ce31f541b4 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:08.669302 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a2b4d-7524-4b29-85a7-38ce31f541b4 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:08.669646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a2b4d-7524-4b29-85a7-38ce31f541b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:08.684013 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 998/2018] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:20:08 2022] POST /identity/v3/projects => generated 409 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:20:08.685018 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f2d26d-fea3-48d6-b877-8922f3881713 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:08.686744 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a8f2d26d-fea3-48d6-b877-8922f3881713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:08.694426 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a57a2b4d-7524-4b29-85a7-38ce31f541b4 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:08.695745 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a57a2b4d-7524-4b29-85a7-38ce31f541b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:08.770586 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1021/2019] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:08.772339 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f2d26d-fea3-48d6-b877-8922f3881713 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:08.776885 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a2b4d-7524-4b29-85a7-38ce31f541b4 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:08.781866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12342aa-be55-4a43-9166-0ae3bb2a3144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:08.782550 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12342aa-be55-4a43-9166-0ae3bb2a3144 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:08.783005 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12342aa-be55-4a43-9166-0ae3bb2a3144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:08.783403 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12342aa-be55-4a43-9166-0ae3bb2a3144 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:08.784066 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12342aa-be55-4a43-9166-0ae3bb2a3144 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:08.785284 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a2b4d-7524-4b29-85a7-38ce31f541b4 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:08.787646 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12342aa-be55-4a43-9166-0ae3bb2a3144 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:08.790437 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a2bc4a-be52-4b46-8e96-b69fd57b69d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:08.791134 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2bc4a-be52-4b46-8e96-b69fd57b69d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:08.791415 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2bc4a-be52-4b46-8e96-b69fd57b69d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:08.791683 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2bc4a-be52-4b46-8e96-b69fd57b69d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:08.792348 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a2bc4a-be52-4b46-8e96-b69fd57b69d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:20:08.792648 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a2bc4a-be52-4b46-8e96-b69fd57b69d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:08.794440 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a2bc4a-be52-4b46-8e96-b69fd57b69d7 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:08.811525 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1022/2020] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:20:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:20:08.816314 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a2bc4a-be52-4b46-8e96-b69fd57b69d7 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:08.821119 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 999/2021] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:20:08 2022] POST /identity/v3/users => generated 311 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:20:08.822471 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12342aa-be55-4a43-9166-0ae3bb2a3144 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:08.827122 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a2bc4a-be52-4b46-8e96-b69fd57b69d7 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:08.834430 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2de1771-4b1d-4959-8ca8-264dfac39fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:08.834430 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de1771-4b1d-4959-8ca8-264dfac39fa6 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:08.834430 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de1771-4b1d-4959-8ca8-264dfac39fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:08.834430 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de1771-4b1d-4959-8ca8-264dfac39fa6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:08.834874 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2de1771-4b1d-4959-8ca8-264dfac39fa6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:08.835867 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2de1771-4b1d-4959-8ca8-264dfac39fa6 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:08.840389 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12342aa-be55-4a43-9166-0ae3bb2a3144 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:08.845702 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06aa64-0db0-4427-abaf-cf0141cf4200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:08.846451 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06aa64-0db0-4427-abaf-cf0141cf4200 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:08.846799 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06aa64-0db0-4427-abaf-cf0141cf4200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:08.847454 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06aa64-0db0-4427-abaf-cf0141cf4200 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:08.847850 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06aa64-0db0-4427-abaf-cf0141cf4200 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:08.849595 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06aa64-0db0-4427-abaf-cf0141cf4200 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:08.867820 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1000/2022] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:08 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:20:08.870162 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06aa64-0db0-4427-abaf-cf0141cf4200 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:08.877187 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1023/2023] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:20:08 2022] POST /identity/v3/projects => generated 371 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:20:08.881749 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2de1771-4b1d-4959-8ca8-264dfac39fa6 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:08.896733 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06aa64-0db0-4427-abaf-cf0141cf4200 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:08.967391 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4fc79f-e71a-43eb-aa3b-41521e781493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:08.968068 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4fc79f-e71a-43eb-aa3b-41521e781493 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:08.968929 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4fc79f-e71a-43eb-aa3b-41521e781493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:08.969205 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4fc79f-e71a-43eb-aa3b-41521e781493 admin admin] PATH_INFO: `/v3/projects/f8505df843904d4aa97771a0a4b4cb5f/users/89f2cc637f044241b4b754cc3871e314/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:08.970481 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2de1771-4b1d-4959-8ca8-264dfac39fa6 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:08.972631 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:08.982848 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7e352-dac5-481d-858c-aceaa99ba0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:08.983678 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7e352-dac5-481d-858c-aceaa99ba0b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:08.984302 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7e352-dac5-481d-858c-aceaa99ba0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:08.984693 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7e352-dac5-481d-858c-aceaa99ba0b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:08.985749 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f7e352-dac5-481d-858c-aceaa99ba0b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:08.988287 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4fc79f-e71a-43eb-aa3b-41521e781493 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8505df843904d4aa97771a0a4b4cb5f, user_id=89f2cc637f044241b4b754cc3871e314, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:08.988800 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f7e352-dac5-481d-858c-aceaa99ba0b4 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:08.992096 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4fc79f-e71a-43eb-aa3b-41521e781493 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:09.009455 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1001/2024] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:20:08 2022] PUT /identity/v3/projects/f8505df843904d4aa97771a0a4b4cb5f/users/89f2cc637f044241b4b754cc3871e314/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:09.013872 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4fc79f-e71a-43eb-aa3b-41521e781493 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:09.015857 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1024/2025] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:20:08 2022] POST /identity/v3/users => generated 286 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:20:09.019814 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7e352-dac5-481d-858c-aceaa99ba0b4 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:09.022444 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:09.032482 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7e352-dac5-481d-858c-aceaa99ba0b4 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:09.090786 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4fc79f-e71a-43eb-aa3b-41521e781493 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:09.096298 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3443957e-fce4-431d-917a-71aa9583e08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:09.096954 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443957e-fce4-431d-917a-71aa9583e08f admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:09.097278 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443957e-fce4-431d-917a-71aa9583e08f admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:09.097549 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443957e-fce4-431d-917a-71aa9583e08f admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:09.098220 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3443957e-fce4-431d-917a-71aa9583e08f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:09.100227 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3443957e-fce4-431d-917a-71aa9583e08f admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:09.105471 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e5d34-0d80-452e-b567-4c6323910992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:09.106487 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e5d34-0d80-452e-b567-4c6323910992 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:09.106603 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e5d34-0d80-452e-b567-4c6323910992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:09.106922 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e5d34-0d80-452e-b567-4c6323910992 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:09.107582 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3e5d34-0d80-452e-b567-4c6323910992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:09.109466 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3e5d34-0d80-452e-b567-4c6323910992 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:09.113686 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1002/2026] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:09 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:20:09.117667 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3443957e-fce4-431d-917a-71aa9583e08f admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:09.122097 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1025/2027] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:09 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 05:20:09.124815 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e5d34-0d80-452e-b567-4c6323910992 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:09.129295 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3443957e-fce4-431d-917a-71aa9583e08f admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:09.134675 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14fcf61-2cf9-4725-b24f-66bcfd561d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:09.135543 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fcf61-2cf9-4725-b24f-66bcfd561d89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:09.135922 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fcf61-2cf9-4725-b24f-66bcfd561d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:09.136176 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e5d34-0d80-452e-b567-4c6323910992 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:09.136295 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fcf61-2cf9-4725-b24f-66bcfd561d89 admin admin] PATH_INFO: `/v3/projects/f8505df843904d4aa97771a0a4b4cb5f/users/89f2cc637f044241b4b754cc3871e314/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:09.138639 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14fcf61-2cf9-4725-b24f-66bcfd561d89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8505df843904d4aa97771a0a4b4cb5f, user_id=89f2cc637f044241b4b754cc3871e314, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:09.141235 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14fcf61-2cf9-4725-b24f-66bcfd561d89 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:09.141341 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46a9fb-9e1a-4ae8-98d7-65fc4a0cc64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:09.142000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46a9fb-9e1a-4ae8-98d7-65fc4a0cc64e admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:09.142407 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46a9fb-9e1a-4ae8-98d7-65fc4a0cc64e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:09.143242 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46a9fb-9e1a-4ae8-98d7-65fc4a0cc64e admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:09.143832 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a46a9fb-9e1a-4ae8-98d7-65fc4a0cc64e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:09.145540 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a46a9fb-9e1a-4ae8-98d7-65fc4a0cc64e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:09.161163 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1026/2028] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 05:20:09.162114 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46a9fb-9e1a-4ae8-98d7-65fc4a0cc64e admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:09.164649 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1003/2029] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:20:09 2022] PUT /identity/v3/projects/f8505df843904d4aa97771a0a4b4cb5f/users/89f2cc637f044241b4b754cc3871e314/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:20:09.176077 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cbdbe0-c843-43d0-ace6-00e3ae2a5137 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:09.176347 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cbdbe0-c843-43d0-ace6-00e3ae2a5137 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:09.176616 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cbdbe0-c843-43d0-ace6-00e3ae2a5137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:09.178779 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46a9fb-9e1a-4ae8-98d7-65fc4a0cc64e admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:09.216784 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-e4cbdbe0-c843-43d0-ace6-00e3ae2a5137 None None] MFA Rules not processed for user `89f2cc637f044241b4b754cc3871e314`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:09.218125 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-e4cbdbe0-c843-43d0-ace6-00e3ae2a5137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:09.266253 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d6e1a0-6b4f-4f06-aa9d-24251d708d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:09.267432 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6e1a0-6b4f-4f06-aa9d-24251d708d18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:09.267889 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6e1a0-6b4f-4f06-aa9d-24251d708d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:09.268933 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6e1a0-6b4f-4f06-aa9d-24251d708d18 admin admin] PATH_INFO: `/v3/projects/3d186b4c3e6b45a1ac80ccf3847fde2a/users/39e7f2867b8f4efca1d3b35bd19db486/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:09.287293 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d6e1a0-6b4f-4f06-aa9d-24251d708d18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d186b4c3e6b45a1ac80ccf3847fde2a, user_id=39e7f2867b8f4efca1d3b35bd19db486, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:09.291231 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d6e1a0-6b4f-4f06-aa9d-24251d708d18 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:09.310042 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1027/2030] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:20:09 2022] PUT /identity/v3/projects/3d186b4c3e6b45a1ac80ccf3847fde2a/users/39e7f2867b8f4efca1d3b35bd19db486/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:20:09.320914 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf2ccc-b032-4737-b5a3-6e00fc8774c9 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:09.321208 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf2ccc-b032-4737-b5a3-6e00fc8774c9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:09.321422 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf2ccc-b032-4737-b5a3-6e00fc8774c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:09.331398 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1004/2031] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:09 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:09.349874 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cbdbe0-c843-43d0-ace6-00e3ae2a5137 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:09.351274 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-e4cbdbe0-c843-43d0-ace6-00e3ae2a5137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:09.365059 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-3baf2ccc-b032-4737-b5a3-6e00fc8774c9 None None] MFA Rules not processed for user `39e7f2867b8f4efca1d3b35bd19db486`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:09.366478 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3baf2ccc-b032-4737-b5a3-6e00fc8774c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:09.414311 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:09.414963 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cbdbe0-c843-43d0-ace6-00e3ae2a5137 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:09.438404 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1028/2032] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:20:09.443316 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baf2ccc-b032-4737-b5a3-6e00fc8774c9 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:09.452366 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:09.491816 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2c0456-259c-4c78-ab77-e237ca0866f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:09.492937 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c0456-259c-4c78-ab77-e237ca0866f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:09.493216 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c0456-259c-4c78-ab77-e237ca0866f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:09.493484 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c0456-259c-4c78-ab77-e237ca0866f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:09.517118 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baf2ccc-b032-4737-b5a3-6e00fc8774c9 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:09.522207 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a7e2b-6e9f-4d40-befc-5285cae30956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:09.522801 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a7e2b-6e9f-4d40-befc-5285cae30956 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:09.523073 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a7e2b-6e9f-4d40-befc-5285cae30956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:09.523417 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a7e2b-6e9f-4d40-befc-5285cae30956 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:09.524676 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5a7e2b-6e9f-4d40-befc-5285cae30956 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:09.526406 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5a7e2b-6e9f-4d40-befc-5285cae30956 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:09.574051 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1029/2033] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:20:09 2022] POST /identity/v3/projects => generated 373 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:20:09.575871 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a7e2b-6e9f-4d40-befc-5285cae30956 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:09.582792 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2c0456-259c-4c78-ab77-e237ca0866f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:09.584297 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2c0456-259c-4c78-ab77-e237ca0866f4 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:09.595079 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1005/2034] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:20:09 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:20:09.690562 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a7e2b-6e9f-4d40-befc-5285cae30956 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:09.712077 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b725f4cf-68bf-4c4d-a986-3c8f7a8dd559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:09.713014 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725f4cf-68bf-4c4d-a986-3c8f7a8dd559 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:09.713298 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725f4cf-68bf-4c4d-a986-3c8f7a8dd559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:09.713568 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725f4cf-68bf-4c4d-a986-3c8f7a8dd559 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:09.714308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b725f4cf-68bf-4c4d-a986-3c8f7a8dd559 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:09.716591 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b725f4cf-68bf-4c4d-a986-3c8f7a8dd559 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:09.748933 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1030/2035] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:20:09 2022] POST /identity/v3/users => generated 293 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:20:09.753549 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2c0456-259c-4c78-ab77-e237ca0866f4 service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:09.765281 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2c0456-259c-4c78-ab77-e237ca0866f4 service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:09.771870 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb502d28-bf72-4f1c-88f2-294b7d4498ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:09.773660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb502d28-bf72-4f1c-88f2-294b7d4498ad admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:09.773940 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb502d28-bf72-4f1c-88f2-294b7d4498ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:09.774232 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb502d28-bf72-4f1c-88f2-294b7d4498ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:09.774979 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb502d28-bf72-4f1c-88f2-294b7d4498ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:09.776527 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb502d28-bf72-4f1c-88f2-294b7d4498ad admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:09.797121 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1006/2036] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:09 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:20:09.801741 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b725f4cf-68bf-4c4d-a986-3c8f7a8dd559 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:09.814607 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b725f4cf-68bf-4c4d-a986-3c8f7a8dd559 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:09.820388 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23797908-f946-47cd-9e73-65c5b39c0223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:09.821173 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23797908-f946-47cd-9e73-65c5b39c0223 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:09.821517 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23797908-f946-47cd-9e73-65c5b39c0223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:09.821901 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23797908-f946-47cd-9e73-65c5b39c0223 admin admin] PATH_INFO: `/v3/projects/441e67c973f940688691d93a55f6fc78/users/e6e242b2981f4f6d9f560d2c3f74b81e/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:09.843371 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23797908-f946-47cd-9e73-65c5b39c0223 admin admin] RBAC: Authorizing `identity:create_grant(project_id=441e67c973f940688691d93a55f6fc78, user_id=e6e242b2981f4f6d9f560d2c3f74b81e, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:09.846503 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23797908-f946-47cd-9e73-65c5b39c0223 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:09.866928 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1031/2037] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:20:09 2022] PUT /identity/v3/projects/441e67c973f940688691d93a55f6fc78/users/e6e242b2981f4f6d9f560d2c3f74b81e/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:09.870938 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb502d28-bf72-4f1c-88f2-294b7d4498ad admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:09.882527 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb502d28-bf72-4f1c-88f2-294b7d4498ad admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:09.888401 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23797908-f946-47cd-9e73-65c5b39c0223 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:09.932826 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:09.950114 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d81f2d-9fa5-4597-b97a-3a0595d9957a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:09.950923 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d81f2d-9fa5-4597-b97a-3a0595d9957a admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:09.951035 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d81f2d-9fa5-4597-b97a-3a0595d9957a admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:09.951258 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d81f2d-9fa5-4597-b97a-3a0595d9957a admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:09.951903 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d81f2d-9fa5-4597-b97a-3a0595d9957a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:09.953522 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d81f2d-9fa5-4597-b97a-3a0595d9957a admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:09.968091 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1007/2038] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:09 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:20:09.969367 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d81f2d-9fa5-4597-b97a-3a0595d9957a admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:09.970883 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23797908-f946-47cd-9e73-65c5b39c0223 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:09.985292 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d81f2d-9fa5-4597-b97a-3a0595d9957a admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:09.986689 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ebfe30-35a0-45d9-8957-4fbfa9cc0df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:09.987675 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebfe30-35a0-45d9-8957-4fbfa9cc0df5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:09.987954 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebfe30-35a0-45d9-8957-4fbfa9cc0df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:09.988291 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebfe30-35a0-45d9-8957-4fbfa9cc0df5 admin admin] PATH_INFO: `/v3/projects/441e67c973f940688691d93a55f6fc78/users/e6e242b2981f4f6d9f560d2c3f74b81e/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:09.990550 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ebfe30-35a0-45d9-8957-4fbfa9cc0df5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=441e67c973f940688691d93a55f6fc78, user_id=e6e242b2981f4f6d9f560d2c3f74b81e, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:09.993311 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ebfe30-35a0-45d9-8957-4fbfa9cc0df5 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:09.995038 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07070ec-a8a4-41cd-bd07-80c0f1eb7a70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:09.995670 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07070ec-a8a4-41cd-bd07-80c0f1eb7a70 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:09.995949 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07070ec-a8a4-41cd-bd07-80c0f1eb7a70 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:09.996214 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07070ec-a8a4-41cd-bd07-80c0f1eb7a70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:10.000596 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c07070ec-a8a4-41cd-bd07-80c0f1eb7a70 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:10.019012 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1008/2039] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:20:09 2022] PUT /identity/v3/projects/441e67c973f940688691d93a55f6fc78/users/e6e242b2981f4f6d9f560d2c3f74b81e/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:10.026116 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7698b-607d-46f7-832b-824cb38981fc None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:10.026116 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7698b-607d-46f7-832b-824cb38981fc None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:10.026116 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7698b-607d-46f7-832b-824cb38981fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:10.065528 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-c3f7698b-607d-46f7-832b-824cb38981fc None None] MFA Rules not processed for user `e6e242b2981f4f6d9f560d2c3f74b81e`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:10.066795 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c3f7698b-607d-46f7-832b-824cb38981fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:10.120761 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07070ec-a8a4-41cd-bd07-80c0f1eb7a70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:10.122600 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07070ec-a8a4-41cd-bd07-80c0f1eb7a70 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:10.137385 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1032/2040] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:20:09 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:20:10.145112 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:10.195148 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1009/2041] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:10 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:10.249139 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07070ec-a8a4-41cd-bd07-80c0f1eb7a70 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:10.250939 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c07070ec-a8a4-41cd-bd07-80c0f1eb7a70 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:10.265920 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07070ec-a8a4-41cd-bd07-80c0f1eb7a70 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:10.338441 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209584e1-d368-4d70-8cf7-18ec069c6f36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:10.339157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209584e1-d368-4d70-8cf7-18ec069c6f36 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:10.339426 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209584e1-d368-4d70-8cf7-18ec069c6f36 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:10.339708 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209584e1-d368-4d70-8cf7-18ec069c6f36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:10.350796 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209584e1-d368-4d70-8cf7-18ec069c6f36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:10.352727 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209584e1-d368-4d70-8cf7-18ec069c6f36 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:10.369610 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1033/2042] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:20:10 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:20:11.188677 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f7698b-607d-46f7-832b-824cb38981fc None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:11.258680 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f7698b-607d-46f7-832b-824cb38981fc None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:11.264572 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab2f3b5-f49f-4681-89ae-d3445db9aa80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:11.265216 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2f3b5-f49f-4681-89ae-d3445db9aa80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:11.265509 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2f3b5-f49f-4681-89ae-d3445db9aa80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:11.265776 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2f3b5-f49f-4681-89ae-d3445db9aa80 admin admin] PATH_INFO: `/v3/users/39e7f2867b8f4efca1d3b35bd19db486` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:11.266923 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab2f3b5-f49f-4681-89ae-d3445db9aa80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39e7f2867b8f4efca1d3b35bd19db486)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:11.268856 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab2f3b5-f49f-4681-89ae-d3445db9aa80 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:11.317734 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-1ab2f3b5-f49f-4681-89ae-d3445db9aa80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39e7f2867b8f4efca1d3b35bd19db486'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:11.325465 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-1ab2f3b5-f49f-4681-89ae-d3445db9aa80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39e7f2867b8f4efca1d3b35bd19db486'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:11.326243 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-1ab2f3b5-f49f-4681-89ae-d3445db9aa80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39e7f2867b8f4efca1d3b35bd19db486'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:11.335180 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-1ab2f3b5-f49f-4681-89ae-d3445db9aa80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39e7f2867b8f4efca1d3b35bd19db486'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:11.335808 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-1ab2f3b5-f49f-4681-89ae-d3445db9aa80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39e7f2867b8f4efca1d3b35bd19db486'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:11.353869 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1010/2043] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:20:11 2022] DELETE /identity/v3/users/39e7f2867b8f4efca1d3b35bd19db486 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:11.358712 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c17615-d42f-4c28-8053-4052bd215afb None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:11.359183 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c17615-d42f-4c28-8053-4052bd215afb None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:11.359399 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c17615-d42f-4c28-8053-4052bd215afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:11.381353 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-61c17615-d42f-4c28-8053-4052bd215afb None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:11.382642 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-61c17615-d42f-4c28-8053-4052bd215afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:11.446467 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1034/2044] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:11.459052 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab2f3b5-f49f-4681-89ae-d3445db9aa80 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:11.461380 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-1ab2f3b5-f49f-4681-89ae-d3445db9aa80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:11.525680 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab2f3b5-f49f-4681-89ae-d3445db9aa80 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:11.529382 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f8439-265e-41d6-a32c-8637ae054834 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:11.530031 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f8439-265e-41d6-a32c-8637ae054834 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:11.530320 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f8439-265e-41d6-a32c-8637ae054834 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:11.530590 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f8439-265e-41d6-a32c-8637ae054834 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:11.540049 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345f8439-265e-41d6-a32c-8637ae054834 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:11.541535 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345f8439-265e-41d6-a32c-8637ae054834 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:11.545518 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:11.553274 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1011/2045] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:20:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:20:11.849286 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c17615-d42f-4c28-8053-4052bd215afb None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:11.850921 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-61c17615-d42f-4c28-8053-4052bd215afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:11.872226 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c17615-d42f-4c28-8053-4052bd215afb None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:11.879637 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c9bee8-46f1-45cc-90b4-eb14936b70b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:11.880955 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c9bee8-46f1-45cc-90b4-eb14936b70b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:11.881499 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c9bee8-46f1-45cc-90b4-eb14936b70b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:11.881912 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c9bee8-46f1-45cc-90b4-eb14936b70b6 admin admin] PATH_INFO: `/v3/projects/3d186b4c3e6b45a1ac80ccf3847fde2a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:11.883192 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c9bee8-46f1-45cc-90b4-eb14936b70b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d186b4c3e6b45a1ac80ccf3847fde2a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:11.885228 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c9bee8-46f1-45cc-90b4-eb14936b70b6 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:11.920336 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:11.939825 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-52c9bee8-46f1-45cc-90b4-eb14936b70b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d186b4c3e6b45a1ac80ccf3847fde2a', 'request_id': 'req-52c9bee8-46f1-45cc-90b4-eb14936b70b6'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:11.946875 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-52c9bee8-46f1-45cc-90b4-eb14936b70b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d186b4c3e6b45a1ac80ccf3847fde2a', 'request_id': 'req-52c9bee8-46f1-45cc-90b4-eb14936b70b6'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:11.956727 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-52c9bee8-46f1-45cc-90b4-eb14936b70b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d186b4c3e6b45a1ac80ccf3847fde2a', 'request_id': 'req-52c9bee8-46f1-45cc-90b4-eb14936b70b6'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:11.976682 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-52c9bee8-46f1-45cc-90b4-eb14936b70b6 admin admin] The token cache is being invalidate because project 3d186b4c3e6b45a1ac80ccf3847fde2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:20:11.976982 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-52c9bee8-46f1-45cc-90b4-eb14936b70b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:11.977790 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-52c9bee8-46f1-45cc-90b4-eb14936b70b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:11.979422 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1035/2046] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:20:11 2022] DELETE /identity/v3/projects/3d186b4c3e6b45a1ac80ccf3847fde2a => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:11.984102 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f8439-265e-41d6-a32c-8637ae054834 service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:12.015165 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f8439-265e-41d6-a32c-8637ae054834 service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:12.097771 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70252f95-c5dd-4b73-b292-c40312160136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:12.098322 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70252f95-c5dd-4b73-b292-c40312160136 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:12.098598 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70252f95-c5dd-4b73-b292-c40312160136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:12.098896 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70252f95-c5dd-4b73-b292-c40312160136 admin admin] PATH_INFO: `/v3/users/e6e242b2981f4f6d9f560d2c3f74b81e` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:12.100155 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70252f95-c5dd-4b73-b292-c40312160136 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6e242b2981f4f6d9f560d2c3f74b81e)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:12.102126 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70252f95-c5dd-4b73-b292-c40312160136 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:12.153404 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-70252f95-c5dd-4b73-b292-c40312160136 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6e242b2981f4f6d9f560d2c3f74b81e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:12.165313 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-70252f95-c5dd-4b73-b292-c40312160136 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6e242b2981f4f6d9f560d2c3f74b81e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:12.166114 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-70252f95-c5dd-4b73-b292-c40312160136 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6e242b2981f4f6d9f560d2c3f74b81e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:12.176401 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-70252f95-c5dd-4b73-b292-c40312160136 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6e242b2981f4f6d9f560d2c3f74b81e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:12.177276 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-70252f95-c5dd-4b73-b292-c40312160136 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6e242b2981f4f6d9f560d2c3f74b81e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:12.199052 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1012/2047] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:20:11 2022] DELETE /identity/v3/users/e6e242b2981f4f6d9f560d2c3f74b81e => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:12.266890 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c9bee8-46f1-45cc-90b4-eb14936b70b6 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:12.268978 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-52c9bee8-46f1-45cc-90b4-eb14936b70b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:12.284516 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c9bee8-46f1-45cc-90b4-eb14936b70b6 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:12.349826 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a4ecb-ead7-4380-98e6-3d5b9e1bc151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:12.350476 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a4ecb-ead7-4380-98e6-3d5b9e1bc151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:12.350749 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a4ecb-ead7-4380-98e6-3d5b9e1bc151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:12.351116 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a4ecb-ead7-4380-98e6-3d5b9e1bc151 admin admin] PATH_INFO: `/v3/users/22b48e6baa1a4c3da5cbcf38e90a5d7d` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:12.352264 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328a4ecb-ead7-4380-98e6-3d5b9e1bc151 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22b48e6baa1a4c3da5cbcf38e90a5d7d)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:12.354335 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328a4ecb-ead7-4380-98e6-3d5b9e1bc151 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:12.405756 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-328a4ecb-ead7-4380-98e6-3d5b9e1bc151 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22b48e6baa1a4c3da5cbcf38e90a5d7d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:12.413654 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-328a4ecb-ead7-4380-98e6-3d5b9e1bc151 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22b48e6baa1a4c3da5cbcf38e90a5d7d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:12.422426 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-328a4ecb-ead7-4380-98e6-3d5b9e1bc151 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22b48e6baa1a4c3da5cbcf38e90a5d7d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:12.423066 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-328a4ecb-ead7-4380-98e6-3d5b9e1bc151 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22b48e6baa1a4c3da5cbcf38e90a5d7d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:12.435240 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-328a4ecb-ead7-4380-98e6-3d5b9e1bc151 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22b48e6baa1a4c3da5cbcf38e90a5d7d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:12.441486 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1036/2048] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:20:12 2022] DELETE /identity/v3/users/22b48e6baa1a4c3da5cbcf38e90a5d7d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:12.446083 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d7066-b524-43fa-abcc-21322c776bbc None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:12.446397 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d7066-b524-43fa-abcc-21322c776bbc None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:12.446693 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d7066-b524-43fa-abcc-21322c776bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:12.467007 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-f35d7066-b524-43fa-abcc-21322c776bbc None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:12.468157 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-f35d7066-b524-43fa-abcc-21322c776bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:12.493888 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a4ecb-ead7-4380-98e6-3d5b9e1bc151 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:12.495370 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-328a4ecb-ead7-4380-98e6-3d5b9e1bc151 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:12.535582 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1013/2049] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:12.546774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35d7066-b524-43fa-abcc-21322c776bbc None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:12.549181 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-f35d7066-b524-43fa-abcc-21322c776bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:12.552916 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:12.569620 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:12.580291 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a4ecb-ead7-4380-98e6-3d5b9e1bc151 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:12.585772 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6c06fc-70a2-4e7d-bc4a-f45395261b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:12.586424 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c06fc-70a2-4e7d-bc4a-f45395261b00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:12.586706 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c06fc-70a2-4e7d-bc4a-f45395261b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:12.586979 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c06fc-70a2-4e7d-bc4a-f45395261b00 admin admin] PATH_INFO: `/v3/projects/441e67c973f940688691d93a55f6fc78` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:12.588094 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6c06fc-70a2-4e7d-bc4a-f45395261b00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=441e67c973f940688691d93a55f6fc78)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:12.590061 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6c06fc-70a2-4e7d-bc4a-f45395261b00 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:12.610692 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35d7066-b524-43fa-abcc-21322c776bbc None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:12.614271 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df27a7bc-4d36-4a60-97a3-5ba545bf2302 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:12.614881 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df27a7bc-4d36-4a60-97a3-5ba545bf2302 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:12.615150 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df27a7bc-4d36-4a60-97a3-5ba545bf2302 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:12.615413 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df27a7bc-4d36-4a60-97a3-5ba545bf2302 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:12.625909 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df27a7bc-4d36-4a60-97a3-5ba545bf2302 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:12.627474 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df27a7bc-4d36-4a60-97a3-5ba545bf2302 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:12.637161 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1014/2050] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:20:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 05:20:12.640052 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3e6c06fc-70a2-4e7d-bc4a-f45395261b00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '441e67c973f940688691d93a55f6fc78', 'request_id': 'req-3e6c06fc-70a2-4e7d-bc4a-f45395261b00'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:12.647197 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3e6c06fc-70a2-4e7d-bc4a-f45395261b00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '441e67c973f940688691d93a55f6fc78', 'request_id': 'req-3e6c06fc-70a2-4e7d-bc4a-f45395261b00'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:12.659800 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3e6c06fc-70a2-4e7d-bc4a-f45395261b00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '441e67c973f940688691d93a55f6fc78', 'request_id': 'req-3e6c06fc-70a2-4e7d-bc4a-f45395261b00'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:12.670732 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3e6c06fc-70a2-4e7d-bc4a-f45395261b00 admin admin] The token cache is being invalidate because project 441e67c973f940688691d93a55f6fc78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:20:12.671095 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3e6c06fc-70a2-4e7d-bc4a-f45395261b00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:12.671817 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3e6c06fc-70a2-4e7d-bc4a-f45395261b00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:12.673575 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1037/2051] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:20:12 2022] DELETE /identity/v3/projects/441e67c973f940688691d93a55f6fc78 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:20:12.678521 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e792cc-eba2-4cbe-8de3-066b1af435bc None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:12.678831 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e792cc-eba2-4cbe-8de3-066b1af435bc None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:12.679102 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e792cc-eba2-4cbe-8de3-066b1af435bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:12.699039 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-32e792cc-eba2-4cbe-8de3-066b1af435bc None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:12.700315 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-32e792cc-eba2-4cbe-8de3-066b1af435bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:12.780611 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1015/2052] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:12.793279 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d82ee-bed3-40bc-b1ab-079fe8028927 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:12.793279 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d82ee-bed3-40bc-b1ab-079fe8028927 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:12.793279 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d82ee-bed3-40bc-b1ab-079fe8028927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:12.818238 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-743d82ee-bed3-40bc-b1ab-079fe8028927 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:12.819509 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-743d82ee-bed3-40bc-b1ab-079fe8028927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:12.827847 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1038/2053] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:12.834107 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e792cc-eba2-4cbe-8de3-066b1af435bc None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:12.844399 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e792cc-eba2-4cbe-8de3-066b1af435bc None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:12.849324 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1472e28d-f6e1-47e6-a36a-2c486b0e0219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:12.849945 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1472e28d-f6e1-47e6-a36a-2c486b0e0219 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:12.850258 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1472e28d-f6e1-47e6-a36a-2c486b0e0219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:12.850521 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1472e28d-f6e1-47e6-a36a-2c486b0e0219 admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:12.851313 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1472e28d-f6e1-47e6-a36a-2c486b0e0219 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:20:12.851823 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1472e28d-f6e1-47e6-a36a-2c486b0e0219 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:12.853458 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1472e28d-f6e1-47e6-a36a-2c486b0e0219 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:12.866256 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1016/2054] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:20:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:20:12.870623 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d82ee-bed3-40bc-b1ab-079fe8028927 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:12.880266 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d82ee-bed3-40bc-b1ab-079fe8028927 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:12.887624 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152a536d-37e4-4cc4-8350-63bd6e9c4c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:12.888238 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a536d-37e4-4cc4-8350-63bd6e9c4c44 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:12.888595 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a536d-37e4-4cc4-8350-63bd6e9c4c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:12.888870 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a536d-37e4-4cc4-8350-63bd6e9c4c44 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:12.889505 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152a536d-37e4-4cc4-8350-63bd6e9c4c44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:12.891606 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152a536d-37e4-4cc4-8350-63bd6e9c4c44 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:12.924402 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1039/2055] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:20:12 2022] POST /identity/v3/projects => generated 369 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:20:12.928903 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1472e28d-f6e1-47e6-a36a-2c486b0e0219 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:12.937188 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152a536d-37e4-4cc4-8350-63bd6e9c4c44 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:12.939412 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-152a536d-37e4-4cc4-8350-63bd6e9c4c44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:12.940755 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1472e28d-f6e1-47e6-a36a-2c486b0e0219 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:13.006717 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2782d3a0-3387-4d83-8a7e-a8fbff95789e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:13.007494 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2782d3a0-3387-4d83-8a7e-a8fbff95789e admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:13.007774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2782d3a0-3387-4d83-8a7e-a8fbff95789e admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:13.008047 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2782d3a0-3387-4d83-8a7e-a8fbff95789e admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:13.008690 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2782d3a0-3387-4d83-8a7e-a8fbff95789e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:13.010713 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2782d3a0-3387-4d83-8a7e-a8fbff95789e admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:13.030065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152a536d-37e4-4cc4-8350-63bd6e9c4c44 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:13.036059 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca0fbc5-707d-486d-9765-6c9d06f912b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:13.036228 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1017/2056] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:20:12 2022] POST /identity/v3/users => generated 285 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:20:13.036800 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0fbc5-707d-486d-9765-6c9d06f912b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:13.037470 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0fbc5-707d-486d-9765-6c9d06f912b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:13.037875 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0fbc5-707d-486d-9765-6c9d06f912b4 admin admin] PATH_INFO: `/v3/projects/e9ced8830de7450dbf632972cc4cd224` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:13.040199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2782d3a0-3387-4d83-8a7e-a8fbff95789e admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:13.040903 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca0fbc5-707d-486d-9765-6c9d06f912b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9ced8830de7450dbf632972cc4cd224)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:13.043107 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca0fbc5-707d-486d-9765-6c9d06f912b4 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:13.055398 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2782d3a0-3387-4d83-8a7e-a8fbff95789e admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:13.065412 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4e68cd-c9b0-419b-b722-65da7f50394f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:13.066036 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e68cd-c9b0-419b-b722-65da7f50394f admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:13.066346 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e68cd-c9b0-419b-b722-65da7f50394f admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:13.066615 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e68cd-c9b0-419b-b722-65da7f50394f admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:13.067256 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4e68cd-c9b0-419b-b722-65da7f50394f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:13.068926 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4e68cd-c9b0-419b-b722-65da7f50394f admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:13.082446 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1018/2057] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:13 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:20:13.086805 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4e68cd-c9b0-419b-b722-65da7f50394f admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:13.097125 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4e68cd-c9b0-419b-b722-65da7f50394f admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:13.098336 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:13.106492 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0ca0fbc5-707d-486d-9765-6c9d06f912b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9ced8830de7450dbf632972cc4cd224', 'request_id': 'req-0ca0fbc5-707d-486d-9765-6c9d06f912b4'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:13.115790 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0ca0fbc5-707d-486d-9765-6c9d06f912b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9ced8830de7450dbf632972cc4cd224', 'request_id': 'req-0ca0fbc5-707d-486d-9765-6c9d06f912b4'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:13.124977 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0ca0fbc5-707d-486d-9765-6c9d06f912b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9ced8830de7450dbf632972cc4cd224', 'request_id': 'req-0ca0fbc5-707d-486d-9765-6c9d06f912b4'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:13.128442 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:13.137242 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0ca0fbc5-707d-486d-9765-6c9d06f912b4 admin admin] The token cache is being invalidate because project e9ced8830de7450dbf632972cc4cd224 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:20:13.137579 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0ca0fbc5-707d-486d-9765-6c9d06f912b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:13.138281 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0ca0fbc5-707d-486d-9765-6c9d06f912b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:13.139988 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1040/2058] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:20:12 2022] DELETE /identity/v3/projects/e9ced8830de7450dbf632972cc4cd224 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:20:13.143556 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca0fbc5-707d-486d-9765-6c9d06f912b4 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:13.144833 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0ca0fbc5-707d-486d-9765-6c9d06f912b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:13.164890 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e0d73-2aa3-449e-ba7f-16578f518c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:13.165506 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e0d73-2aa3-449e-ba7f-16578f518c7f admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:13.165779 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e0d73-2aa3-449e-ba7f-16578f518c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:13.166075 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e0d73-2aa3-449e-ba7f-16578f518c7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:13.166720 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776e0d73-2aa3-449e-ba7f-16578f518c7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:13.168354 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776e0d73-2aa3-449e-ba7f-16578f518c7f admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:13.178167 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1019/2059] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:13 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:20:13.182322 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e0d73-2aa3-449e-ba7f-16578f518c7f admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:13.183945 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-776e0d73-2aa3-449e-ba7f-16578f518c7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:13.195817 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e0d73-2aa3-449e-ba7f-16578f518c7f admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:13.200427 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b9ce80-ab3d-41a3-a0be-8429cc5cebc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:13.201171 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9ce80-ab3d-41a3-a0be-8429cc5cebc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:13.201451 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9ce80-ab3d-41a3-a0be-8429cc5cebc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:13.201718 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9ce80-ab3d-41a3-a0be-8429cc5cebc5 admin admin] PATH_INFO: `/v3/projects/2e3e6a92625347609c46698944fa41f6/users/9d44e4c0004343709283182bbd1358c8/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:13.213057 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca0fbc5-707d-486d-9765-6c9d06f912b4 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:13.218017 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3e583-3e43-4dd1-baf8-97b7e2c56d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:13.218733 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3e583-3e43-4dd1-baf8-97b7e2c56d31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:13.218972 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3e583-3e43-4dd1-baf8-97b7e2c56d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:13.219227 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3e583-3e43-4dd1-baf8-97b7e2c56d31 admin admin] PATH_INFO: `/v3/users/b33d190ced304b16a2735464ad50d12f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:13.220226 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b9ce80-ab3d-41a3-a0be-8429cc5cebc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e3e6a92625347609c46698944fa41f6, user_id=9d44e4c0004343709283182bbd1358c8, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:13.220349 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b3e583-3e43-4dd1-baf8-97b7e2c56d31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b33d190ced304b16a2735464ad50d12f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:13.222351 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b3e583-3e43-4dd1-baf8-97b7e2c56d31 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:13.223001 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b9ce80-ab3d-41a3-a0be-8429cc5cebc5 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:13.239290 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1020/2060] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:20:13 2022] PUT /identity/v3/projects/2e3e6a92625347609c46698944fa41f6/users/9d44e4c0004343709283182bbd1358c8/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:20:13.251174 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7ce02-9956-4a7e-908f-aed50697e775 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:13.251471 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7ce02-9956-4a7e-908f-aed50697e775 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:13.251759 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7ce02-9956-4a7e-908f-aed50697e775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:13.273067 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e5b3e583-3e43-4dd1-baf8-97b7e2c56d31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b33d190ced304b16a2735464ad50d12f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:13.280329 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e5b3e583-3e43-4dd1-baf8-97b7e2c56d31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b33d190ced304b16a2735464ad50d12f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:13.287293 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-5fe7ce02-9956-4a7e-908f-aed50697e775 None None] MFA Rules not processed for user `9d44e4c0004343709283182bbd1358c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:13.288861 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-5fe7ce02-9956-4a7e-908f-aed50697e775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:13.288999 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e5b3e583-3e43-4dd1-baf8-97b7e2c56d31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b33d190ced304b16a2735464ad50d12f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:13.289636 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e5b3e583-3e43-4dd1-baf8-97b7e2c56d31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b33d190ced304b16a2735464ad50d12f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:13.302717 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e5b3e583-3e43-4dd1-baf8-97b7e2c56d31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b33d190ced304b16a2735464ad50d12f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:13.308837 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1041/2061] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:20:13 2022] DELETE /identity/v3/users/b33d190ced304b16a2735464ad50d12f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:20:13.365336 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1021/2062] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:20:13.370803 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3e583-3e43-4dd1-baf8-97b7e2c56d31 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:13.373592 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e5b3e583-3e43-4dd1-baf8-97b7e2c56d31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:13.428606 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:13.449746 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3e583-3e43-4dd1-baf8-97b7e2c56d31 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:13.454547 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1ea82f-bdb1-44f5-ae5a-214b7fe27af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:13.455233 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ea82f-bdb1-44f5-ae5a-214b7fe27af6 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:13.455578 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ea82f-bdb1-44f5-ae5a-214b7fe27af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:13.455910 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ea82f-bdb1-44f5-ae5a-214b7fe27af6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:13.456610 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1ea82f-bdb1-44f5-ae5a-214b7fe27af6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:13.458572 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1ea82f-bdb1-44f5-ae5a-214b7fe27af6 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:13.489816 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1042/2063] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:20:13 2022] POST /identity/v3/projects => generated 369 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:20:13.493941 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7ce02-9956-4a7e-908f-aed50697e775 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:13.563901 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7ce02-9956-4a7e-908f-aed50697e775 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:13.594450 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1ea82f-bdb1-44f5-ae5a-214b7fe27af6 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:13.596606 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4f1ea82f-bdb1-44f5-ae5a-214b7fe27af6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:13.603505 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:13.656345 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbb6cca-9faf-4da8-b372-1e3b9a205ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:13.657240 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb6cca-9faf-4da8-b372-1e3b9a205ae1 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:13.658325 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb6cca-9faf-4da8-b372-1e3b9a205ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:13.658774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb6cca-9faf-4da8-b372-1e3b9a205ae1 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:13.659447 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbb6cca-9faf-4da8-b372-1e3b9a205ae1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:13.661378 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbb6cca-9faf-4da8-b372-1e3b9a205ae1 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:13.692278 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1ea82f-bdb1-44f5-ae5a-214b7fe27af6 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:13.692433 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1022/2064] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:20:13 2022] POST /identity/v3/users => generated 291 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:20:13.693573 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbb6cca-9faf-4da8-b372-1e3b9a205ae1 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:13.695216 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1727e02-8193-4534-b955-58fb9cfd844c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:13.695935 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1727e02-8193-4534-b955-58fb9cfd844c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:13.696217 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1727e02-8193-4534-b955-58fb9cfd844c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:13.696498 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1727e02-8193-4534-b955-58fb9cfd844c admin admin] PATH_INFO: `/v3/projects/16c9d6e4f76849d1b6b7512b8b54df97` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:13.697623 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1727e02-8193-4534-b955-58fb9cfd844c admin admin] RBAC: Authorizing `identity:delete_project(project_id=16c9d6e4f76849d1b6b7512b8b54df97)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:13.699760 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1727e02-8193-4534-b955-58fb9cfd844c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:13.704106 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbb6cca-9faf-4da8-b372-1e3b9a205ae1 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:13.713832 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b26530f-35f9-4f95-8c2a-21ecc2974f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:13.714546 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26530f-35f9-4f95-8c2a-21ecc2974f06 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:13.714921 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26530f-35f9-4f95-8c2a-21ecc2974f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:13.715320 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26530f-35f9-4f95-8c2a-21ecc2974f06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:13.716108 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b26530f-35f9-4f95-8c2a-21ecc2974f06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:13.717820 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b26530f-35f9-4f95-8c2a-21ecc2974f06 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:13.729128 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1023/2065] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 05:20:13.733373 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b26530f-35f9-4f95-8c2a-21ecc2974f06 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:13.743592 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b26530f-35f9-4f95-8c2a-21ecc2974f06 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:13.750812 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-f1727e02-8193-4534-b955-58fb9cfd844c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16c9d6e4f76849d1b6b7512b8b54df97', 'request_id': 'req-f1727e02-8193-4534-b955-58fb9cfd844c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:13.756972 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-f1727e02-8193-4534-b955-58fb9cfd844c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16c9d6e4f76849d1b6b7512b8b54df97', 'request_id': 'req-f1727e02-8193-4534-b955-58fb9cfd844c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:13.769830 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-f1727e02-8193-4534-b955-58fb9cfd844c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16c9d6e4f76849d1b6b7512b8b54df97', 'request_id': 'req-f1727e02-8193-4534-b955-58fb9cfd844c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:13.781068 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-f1727e02-8193-4534-b955-58fb9cfd844c admin admin] The token cache is being invalidate because project 16c9d6e4f76849d1b6b7512b8b54df97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:20:13.781351 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-f1727e02-8193-4534-b955-58fb9cfd844c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:13.782026 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-f1727e02-8193-4534-b955-58fb9cfd844c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:13.783546 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1043/2066] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:20:13 2022] DELETE /identity/v3/projects/16c9d6e4f76849d1b6b7512b8b54df97 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:13.788511 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41fd87e-7b61-4915-bfdb-905559133d87 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:13.788728 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41fd87e-7b61-4915-bfdb-905559133d87 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:13.788994 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41fd87e-7b61-4915-bfdb-905559133d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:13.805769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13651c02-eb3d-4c3a-b4db-fe376498de36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:13.806743 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13651c02-eb3d-4c3a-b4db-fe376498de36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:13.806743 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13651c02-eb3d-4c3a-b4db-fe376498de36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:13.807064 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13651c02-eb3d-4c3a-b4db-fe376498de36 admin admin] PATH_INFO: `/v3/projects/6c912ecd1bda4a45b70176da39c19e1e/users/5e6907e1c135464eac35e68f1c1720de/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:13.809807 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a41fd87e-7b61-4915-bfdb-905559133d87 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:13.812053 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a41fd87e-7b61-4915-bfdb-905559133d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:13.820492 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1044/2067] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:20:13.823550 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13651c02-eb3d-4c3a-b4db-fe376498de36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c912ecd1bda4a45b70176da39c19e1e, user_id=5e6907e1c135464eac35e68f1c1720de, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:13.826440 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13651c02-eb3d-4c3a-b4db-fe376498de36 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:13.829102 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f95c58d-4b0c-4aa3-9478-b225c9569749 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:13.829378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f95c58d-4b0c-4aa3-9478-b225c9569749 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:13.829644 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f95c58d-4b0c-4aa3-9478-b225c9569749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:13.843337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1024/2068] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:20:13 2022] PUT /identity/v3/projects/6c912ecd1bda4a45b70176da39c19e1e/users/5e6907e1c135464eac35e68f1c1720de/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:13.848540 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13651c02-eb3d-4c3a-b4db-fe376498de36 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:13.854087 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-13651c02-eb3d-4c3a-b4db-fe376498de36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:13.855542 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-4f95c58d-4b0c-4aa3-9478-b225c9569749 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:13.856941 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4f95c58d-4b0c-4aa3-9478-b225c9569749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:13.910037 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:13.925054 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1045/2069] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:20:13.929058 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f95c58d-4b0c-4aa3-9478-b225c9569749 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:13.934796 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13651c02-eb3d-4c3a-b4db-fe376498de36 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:13.939540 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca9a2f-9268-4186-be9f-5f2d92362a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:13.940145 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca9a2f-9268-4186-be9f-5f2d92362a78 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:13.940422 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca9a2f-9268-4186-be9f-5f2d92362a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:13.940703 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca9a2f-9268-4186-be9f-5f2d92362a78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:13.940903 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f95c58d-4b0c-4aa3-9478-b225c9569749 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:13.941346 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ca9a2f-9268-4186-be9f-5f2d92362a78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:13.943034 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ca9a2f-9268-4186-be9f-5f2d92362a78 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:13.947223 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2786f1-6917-43ae-9a12-54f7aaa0321f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:13.947855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2786f1-6917-43ae-9a12-54f7aaa0321f admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:13.948132 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2786f1-6917-43ae-9a12-54f7aaa0321f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:13.948403 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2786f1-6917-43ae-9a12-54f7aaa0321f admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:13.949054 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2786f1-6917-43ae-9a12-54f7aaa0321f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:20:13.949376 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2786f1-6917-43ae-9a12-54f7aaa0321f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:13.951049 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2786f1-6917-43ae-9a12-54f7aaa0321f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:13.953672 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1025/2070] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:13 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 05:20:13.958020 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca9a2f-9268-4186-be9f-5f2d92362a78 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:13.964026 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1046/2071] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:20:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:20:13.968524 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2786f1-6917-43ae-9a12-54f7aaa0321f admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:13.969825 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca9a2f-9268-4186-be9f-5f2d92362a78 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:13.975073 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c055bb2-9df5-4b67-927d-f10bb737f1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:13.975722 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c055bb2-9df5-4b67-927d-f10bb737f1b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:13.975999 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c055bb2-9df5-4b67-927d-f10bb737f1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:13.976268 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c055bb2-9df5-4b67-927d-f10bb737f1b9 admin admin] PATH_INFO: `/v3/projects/6c912ecd1bda4a45b70176da39c19e1e/users/5e6907e1c135464eac35e68f1c1720de/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:13.978211 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c055bb2-9df5-4b67-927d-f10bb737f1b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c912ecd1bda4a45b70176da39c19e1e, user_id=5e6907e1c135464eac35e68f1c1720de, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:13.978802 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2786f1-6917-43ae-9a12-54f7aaa0321f admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:13.980872 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c055bb2-9df5-4b67-927d-f10bb737f1b9 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:13.983638 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e8be87-6a78-46d2-a0f8-a98f2a35c9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:13.984238 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8be87-6a78-46d2-a0f8-a98f2a35c9d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:13.984515 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8be87-6a78-46d2-a0f8-a98f2a35c9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:13.985102 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8be87-6a78-46d2-a0f8-a98f2a35c9d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:13.985792 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e8be87-6a78-46d2-a0f8-a98f2a35c9d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:13.987713 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e8be87-6a78-46d2-a0f8-a98f2a35c9d0 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:13.992851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1026/2072] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:20:13 2022] PUT /identity/v3/projects/6c912ecd1bda4a45b70176da39c19e1e/users/5e6907e1c135464eac35e68f1c1720de/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:20:14.003211 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ea092-9393-4098-b6d0-0881a52568ef None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:14.003487 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ea092-9393-4098-b6d0-0881a52568ef None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:14.003755 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ea092-9393-4098-b6d0-0881a52568ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:14.017920 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1047/2073] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:20:13 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:20:14.021986 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e8be87-6a78-46d2-a0f8-a98f2a35c9d0 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:14.033415 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-cb3ea092-9393-4098-b6d0-0881a52568ef None None] MFA Rules not processed for user `5e6907e1c135464eac35e68f1c1720de`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:14.034776 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-cb3ea092-9393-4098-b6d0-0881a52568ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:14.044166 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:14.093952 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e8be87-6a78-46d2-a0f8-a98f2a35c9d0 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:14.106287 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80311f20-4c9a-45fd-b56a-bd478157a40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:14.106948 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80311f20-4c9a-45fd-b56a-bd478157a40c admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:14.107228 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80311f20-4c9a-45fd-b56a-bd478157a40c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:14.107480 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80311f20-4c9a-45fd-b56a-bd478157a40c admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:14.108142 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80311f20-4c9a-45fd-b56a-bd478157a40c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:14.110442 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80311f20-4c9a-45fd-b56a-bd478157a40c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:14.138871 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1027/2074] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:14 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:20:14.145476 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1048/2075] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:20:14 2022] POST /identity/v3/users => generated 287 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:20:14.149272 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3ea092-9393-4098-b6d0-0881a52568ef None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:14.149553 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80311f20-4c9a-45fd-b56a-bd478157a40c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:14.151308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-80311f20-4c9a-45fd-b56a-bd478157a40c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:14.159977 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3ea092-9393-4098-b6d0-0881a52568ef None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:14.165407 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb7cba-f441-494e-ba44-f8cd4c679d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:14.166055 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb7cba-f441-494e-ba44-f8cd4c679d8f admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:14.166345 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb7cba-f441-494e-ba44-f8cd4c679d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:14.166620 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb7cba-f441-494e-ba44-f8cd4c679d8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:14.167287 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abb7cba-f441-494e-ba44-f8cd4c679d8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:14.168997 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abb7cba-f441-494e-ba44-f8cd4c679d8f admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:14.179684 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1028/2076] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:20:14.183775 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb7cba-f441-494e-ba44-f8cd4c679d8f admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:14.193755 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb7cba-f441-494e-ba44-f8cd4c679d8f admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:14.198582 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eb075b-61ff-43e5-acd0-4546a578361b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:14.199176 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eb075b-61ff-43e5-acd0-4546a578361b admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:14.199460 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eb075b-61ff-43e5-acd0-4546a578361b admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:14.199729 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eb075b-61ff-43e5-acd0-4546a578361b admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:14.200447 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eb075b-61ff-43e5-acd0-4546a578361b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:14.202056 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eb075b-61ff-43e5-acd0-4546a578361b admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:14.212357 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80311f20-4c9a-45fd-b56a-bd478157a40c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:14.213326 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1029/2077] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 05:20:14.216183 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c12f63e-ca8b-48a8-b2b8-46f5a0c8ff72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:14.216808 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12f63e-ca8b-48a8-b2b8-46f5a0c8ff72 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:14.217084 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12f63e-ca8b-48a8-b2b8-46f5a0c8ff72 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:14.217360 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12f63e-ca8b-48a8-b2b8-46f5a0c8ff72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:14.217553 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eb075b-61ff-43e5-acd0-4546a578361b admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:14.227087 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c12f63e-ca8b-48a8-b2b8-46f5a0c8ff72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:14.228011 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eb075b-61ff-43e5-acd0-4546a578361b admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:14.228569 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c12f63e-ca8b-48a8-b2b8-46f5a0c8ff72 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:14.233025 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff615f-56d5-4dc3-8a70-27f5e282519c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:14.233674 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff615f-56d5-4dc3-8a70-27f5e282519c admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:14.233950 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff615f-56d5-4dc3-8a70-27f5e282519c admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:14.234253 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff615f-56d5-4dc3-8a70-27f5e282519c admin admin] PATH_INFO: `/v3/projects/8b3cfc1ea7e74cd3b92f51e8c56b8ed8/users/ba4f2506c8e540bd84e584c7d1bc838d/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:14.238928 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1049/2078] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:20:14 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 05:20:14.249634 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cff615f-56d5-4dc3-8a70-27f5e282519c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b3cfc1ea7e74cd3b92f51e8c56b8ed8, user_id=ba4f2506c8e540bd84e584c7d1bc838d, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:14.252678 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cff615f-56d5-4dc3-8a70-27f5e282519c admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:14.271735 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1030/2079] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:20:14 2022] PUT /identity/v3/projects/8b3cfc1ea7e74cd3b92f51e8c56b8ed8/users/ba4f2506c8e540bd84e584c7d1bc838d/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:20:14.285182 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2698b-ac14-46f7-8fe7-6fefc7eef137 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:14.285495 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2698b-ac14-46f7-8fe7-6fefc7eef137 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:14.285743 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2698b-ac14-46f7-8fe7-6fefc7eef137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:14.325479 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b2a2698b-ac14-46f7-8fe7-6fefc7eef137 None None] MFA Rules not processed for user `ba4f2506c8e540bd84e584c7d1bc838d`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:14.325479 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b2a2698b-ac14-46f7-8fe7-6fefc7eef137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:14.369080 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:14.395627 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1050/2080] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:14.400271 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff615f-56d5-4dc3-8a70-27f5e282519c admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:14.474209 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff615f-56d5-4dc3-8a70-27f5e282519c admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:14.478991 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc07374-0812-4bbd-80ce-68a32789f8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:14.479594 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc07374-0812-4bbd-80ce-68a32789f8fa admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:14.479876 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc07374-0812-4bbd-80ce-68a32789f8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:14.480143 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc07374-0812-4bbd-80ce-68a32789f8fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:14.480807 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc07374-0812-4bbd-80ce-68a32789f8fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:14.482788 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc07374-0812-4bbd-80ce-68a32789f8fa admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:14.515726 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1031/2081] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:20:14 2022] POST /identity/v3/projects => generated 373 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:20:14.516871 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2698b-ac14-46f7-8fe7-6fefc7eef137 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:14.528643 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2698b-ac14-46f7-8fe7-6fefc7eef137 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:14.592692 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95688572-387c-4a02-9377-4ff2e5197189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:14.593577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95688572-387c-4a02-9377-4ff2e5197189 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:14.593825 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95688572-387c-4a02-9377-4ff2e5197189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:14.594132 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95688572-387c-4a02-9377-4ff2e5197189 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:14.594778 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95688572-387c-4a02-9377-4ff2e5197189 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:14.596746 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95688572-387c-4a02-9377-4ff2e5197189 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:14.619855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1051/2082] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:20:14 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:20:14.623647 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc07374-0812-4bbd-80ce-68a32789f8fa admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:14.634728 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc07374-0812-4bbd-80ce-68a32789f8fa admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:14.644748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c346bc-1e93-4aec-908e-9b89329caea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:14.645382 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c346bc-1e93-4aec-908e-9b89329caea1 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:14.646114 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c346bc-1e93-4aec-908e-9b89329caea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:14.646114 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c346bc-1e93-4aec-908e-9b89329caea1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:14.646629 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c346bc-1e93-4aec-908e-9b89329caea1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:14.648260 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c346bc-1e93-4aec-908e-9b89329caea1 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:14.661162 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1032/2083] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:20:14.665188 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95688572-387c-4a02-9377-4ff2e5197189 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:14.675400 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95688572-387c-4a02-9377-4ff2e5197189 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:14.681023 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8876ee96-4c47-46d3-9003-d3def506ad45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:14.681696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876ee96-4c47-46d3-9003-d3def506ad45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:14.681984 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876ee96-4c47-46d3-9003-d3def506ad45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:14.682296 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876ee96-4c47-46d3-9003-d3def506ad45 admin admin] PATH_INFO: `/v3/projects/85edb18dc5b8448ca3ade80eaa87d45d/users/4726edc6dd144d719a46572fa60a6d6e/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:14.702784 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8876ee96-4c47-46d3-9003-d3def506ad45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85edb18dc5b8448ca3ade80eaa87d45d, user_id=4726edc6dd144d719a46572fa60a6d6e, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:14.705489 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8876ee96-4c47-46d3-9003-d3def506ad45 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:14.721763 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1052/2084] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:20:14 2022] PUT /identity/v3/projects/85edb18dc5b8448ca3ade80eaa87d45d/users/4726edc6dd144d719a46572fa60a6d6e/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:14.727486 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c346bc-1e93-4aec-908e-9b89329caea1 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:14.739358 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c346bc-1e93-4aec-908e-9b89329caea1 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:14.777410 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:14.807982 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175bfac7-2fbb-43c4-b8c0-236c2e2648ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:14.808674 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175bfac7-2fbb-43c4-b8c0-236c2e2648ff admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:14.809054 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175bfac7-2fbb-43c4-b8c0-236c2e2648ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:14.809467 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175bfac7-2fbb-43c4-b8c0-236c2e2648ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:14.810282 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175bfac7-2fbb-43c4-b8c0-236c2e2648ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:14.812110 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175bfac7-2fbb-43c4-b8c0-236c2e2648ff admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:14.828507 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1033/2085] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:14 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:20:14.829593 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8876ee96-4c47-46d3-9003-d3def506ad45 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:14.840051 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8876ee96-4c47-46d3-9003-d3def506ad45 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:14.844956 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ea72d-9804-4012-b74b-a9e0c3b1fa3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:14.845633 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ea72d-9804-4012-b74b-a9e0c3b1fa3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:14.845909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ea72d-9804-4012-b74b-a9e0c3b1fa3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:14.846206 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ea72d-9804-4012-b74b-a9e0c3b1fa3a admin admin] PATH_INFO: `/v3/projects/85edb18dc5b8448ca3ade80eaa87d45d/users/4726edc6dd144d719a46572fa60a6d6e/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:14.850933 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ea72d-9804-4012-b74b-a9e0c3b1fa3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=85edb18dc5b8448ca3ade80eaa87d45d, user_id=4726edc6dd144d719a46572fa60a6d6e, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:14.853928 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ea72d-9804-4012-b74b-a9e0c3b1fa3a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:14.865495 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1053/2086] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:20:14 2022] PUT /identity/v3/projects/85edb18dc5b8448ca3ade80eaa87d45d/users/4726edc6dd144d719a46572fa60a6d6e/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:20:14.876004 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08291c24-72d0-4481-a051-c75414890910 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:14.876389 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08291c24-72d0-4481-a051-c75414890910 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:14.876774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08291c24-72d0-4481-a051-c75414890910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:14.915320 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-08291c24-72d0-4481-a051-c75414890910 None None] MFA Rules not processed for user `4726edc6dd144d719a46572fa60a6d6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:14.916627 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-08291c24-72d0-4481-a051-c75414890910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:15.001329 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1034/2087] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:14 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:20:15.013090 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ea72d-9804-4012-b74b-a9e0c3b1fa3a admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:15.014741 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-660ea72d-9804-4012-b74b-a9e0c3b1fa3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:15.080012 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ea72d-9804-4012-b74b-a9e0c3b1fa3a admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:15.083640 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa27e5b-5a77-46b7-8ed5-7d635f03bc27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:15.084245 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa27e5b-5a77-46b7-8ed5-7d635f03bc27 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:15.084520 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa27e5b-5a77-46b7-8ed5-7d635f03bc27 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:15.084782 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa27e5b-5a77-46b7-8ed5-7d635f03bc27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:15.135352 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:15.147613 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa27e5b-5a77-46b7-8ed5-7d635f03bc27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:15.149443 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa27e5b-5a77-46b7-8ed5-7d635f03bc27 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:15.158913 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1054/2088] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:20:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:20:15.817996 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08291c24-72d0-4481-a051-c75414890910 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:15.882664 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08291c24-72d0-4481-a051-c75414890910 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:15.887352 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a388e2-a1df-474c-bbae-799e99155663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:15.887985 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a388e2-a1df-474c-bbae-799e99155663 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:15.888255 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a388e2-a1df-474c-bbae-799e99155663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:15.888521 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a388e2-a1df-474c-bbae-799e99155663 admin admin] PATH_INFO: `/v3/users/9d44e4c0004343709283182bbd1358c8` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:15.889696 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a388e2-a1df-474c-bbae-799e99155663 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d44e4c0004343709283182bbd1358c8)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:15.891663 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a388e2-a1df-474c-bbae-799e99155663 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:15.945847 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-63a388e2-a1df-474c-bbae-799e99155663 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d44e4c0004343709283182bbd1358c8'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:15.953335 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-63a388e2-a1df-474c-bbae-799e99155663 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d44e4c0004343709283182bbd1358c8'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:15.954047 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-63a388e2-a1df-474c-bbae-799e99155663 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d44e4c0004343709283182bbd1358c8'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:15.955694 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:15.963567 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-63a388e2-a1df-474c-bbae-799e99155663 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d44e4c0004343709283182bbd1358c8'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:15.964737 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-63a388e2-a1df-474c-bbae-799e99155663 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d44e4c0004343709283182bbd1358c8'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:15.984416 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1035/2089] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:20:15 2022] DELETE /identity/v3/users/9d44e4c0004343709283182bbd1358c8 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:15.989299 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c3784-511d-4b16-b6fc-9fb3bcb1df9c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:15.989822 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c3784-511d-4b16-b6fc-9fb3bcb1df9c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:15.989822 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c3784-511d-4b16-b6fc-9fb3bcb1df9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:16.010078 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-8c2c3784-511d-4b16-b6fc-9fb3bcb1df9c None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:16.011558 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8c2c3784-511d-4b16-b6fc-9fb3bcb1df9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:16.078040 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1055/2090] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:20:16.089065 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a388e2-a1df-474c-bbae-799e99155663 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:16.090879 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-63a388e2-a1df-474c-bbae-799e99155663 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:16.155818 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a388e2-a1df-474c-bbae-799e99155663 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:16.160246 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b2d6ea-7c7a-41f5-8e85-eeeafc9e7a96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:16.160873 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2d6ea-7c7a-41f5-8e85-eeeafc9e7a96 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:16.161126 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2d6ea-7c7a-41f5-8e85-eeeafc9e7a96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:16.161399 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2d6ea-7c7a-41f5-8e85-eeeafc9e7a96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:16.171414 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b2d6ea-7c7a-41f5-8e85-eeeafc9e7a96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:16.172887 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b2d6ea-7c7a-41f5-8e85-eeeafc9e7a96 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:16.183497 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1036/2091] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:20:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:20:16.469015 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2c3784-511d-4b16-b6fc-9fb3bcb1df9c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:16.470862 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8c2c3784-511d-4b16-b6fc-9fb3bcb1df9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:16.483566 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2c3784-511d-4b16-b6fc-9fb3bcb1df9c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:16.489260 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f38646f-8cba-4b77-ae78-20cfffccd852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:16.489905 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38646f-8cba-4b77-ae78-20cfffccd852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:16.490217 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38646f-8cba-4b77-ae78-20cfffccd852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:16.490487 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38646f-8cba-4b77-ae78-20cfffccd852 admin admin] PATH_INFO: `/v3/projects/2e3e6a92625347609c46698944fa41f6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:16.491621 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f38646f-8cba-4b77-ae78-20cfffccd852 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e3e6a92625347609c46698944fa41f6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:16.493664 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f38646f-8cba-4b77-ae78-20cfffccd852 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:16.544566 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1f38646f-8cba-4b77-ae78-20cfffccd852 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e3e6a92625347609c46698944fa41f6', 'request_id': 'req-1f38646f-8cba-4b77-ae78-20cfffccd852'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:16.550914 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1f38646f-8cba-4b77-ae78-20cfffccd852 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e3e6a92625347609c46698944fa41f6', 'request_id': 'req-1f38646f-8cba-4b77-ae78-20cfffccd852'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:16.559369 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1f38646f-8cba-4b77-ae78-20cfffccd852 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e3e6a92625347609c46698944fa41f6', 'request_id': 'req-1f38646f-8cba-4b77-ae78-20cfffccd852'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:16.569619 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1f38646f-8cba-4b77-ae78-20cfffccd852 admin admin] The token cache is being invalidate because project 2e3e6a92625347609c46698944fa41f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:20:16.569896 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1f38646f-8cba-4b77-ae78-20cfffccd852 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:16.570536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1f38646f-8cba-4b77-ae78-20cfffccd852 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:16.572008 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1056/2092] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:20:16 2022] DELETE /identity/v3/projects/2e3e6a92625347609c46698944fa41f6 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:16.575951 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b2d6ea-7c7a-41f5-8e85-eeeafc9e7a96 service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:16.586530 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b2d6ea-7c7a-41f5-8e85-eeeafc9e7a96 service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:16.644763 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6db96-bc37-418e-9214-361de49e6ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:16.645561 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6db96-bc37-418e-9214-361de49e6ae3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:16.645674 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6db96-bc37-418e-9214-361de49e6ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:16.645897 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6db96-bc37-418e-9214-361de49e6ae3 admin admin] PATH_INFO: `/v3/users/5e6907e1c135464eac35e68f1c1720de` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:16.647075 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc6db96-bc37-418e-9214-361de49e6ae3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e6907e1c135464eac35e68f1c1720de)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:16.649016 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc6db96-bc37-418e-9214-361de49e6ae3 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:16.696689 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ebc6db96-bc37-418e-9214-361de49e6ae3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e6907e1c135464eac35e68f1c1720de'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:16.703531 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ebc6db96-bc37-418e-9214-361de49e6ae3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e6907e1c135464eac35e68f1c1720de'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:16.704173 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ebc6db96-bc37-418e-9214-361de49e6ae3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e6907e1c135464eac35e68f1c1720de'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:16.711782 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ebc6db96-bc37-418e-9214-361de49e6ae3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e6907e1c135464eac35e68f1c1720de'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:16.712540 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ebc6db96-bc37-418e-9214-361de49e6ae3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e6907e1c135464eac35e68f1c1720de'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:16.723796 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:16.732261 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1037/2093] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:20:16 2022] DELETE /identity/v3/users/5e6907e1c135464eac35e68f1c1720de => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:20:17.005341 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f38646f-8cba-4b77-ae78-20cfffccd852 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:17.006914 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1f38646f-8cba-4b77-ae78-20cfffccd852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:17.020177 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f38646f-8cba-4b77-ae78-20cfffccd852 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:17.086372 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c9209-5da7-4348-b2e9-fc37bc2c7984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:17.087013 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c9209-5da7-4348-b2e9-fc37bc2c7984 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:17.087313 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c9209-5da7-4348-b2e9-fc37bc2c7984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:17.087581 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c9209-5da7-4348-b2e9-fc37bc2c7984 admin admin] PATH_INFO: `/v3/projects/6c912ecd1bda4a45b70176da39c19e1e` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:17.088716 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c9209-5da7-4348-b2e9-fc37bc2c7984 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c912ecd1bda4a45b70176da39c19e1e)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:17.090663 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c9209-5da7-4348-b2e9-fc37bc2c7984 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:17.106171 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:17.139575 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-469c9209-5da7-4348-b2e9-fc37bc2c7984 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c912ecd1bda4a45b70176da39c19e1e', 'request_id': 'req-469c9209-5da7-4348-b2e9-fc37bc2c7984'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:17.146183 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-469c9209-5da7-4348-b2e9-fc37bc2c7984 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c912ecd1bda4a45b70176da39c19e1e', 'request_id': 'req-469c9209-5da7-4348-b2e9-fc37bc2c7984'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:17.155934 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-469c9209-5da7-4348-b2e9-fc37bc2c7984 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c912ecd1bda4a45b70176da39c19e1e', 'request_id': 'req-469c9209-5da7-4348-b2e9-fc37bc2c7984'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:17.169462 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-469c9209-5da7-4348-b2e9-fc37bc2c7984 admin admin] The token cache is being invalidate because project 6c912ecd1bda4a45b70176da39c19e1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:20:17.169745 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-469c9209-5da7-4348-b2e9-fc37bc2c7984 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:17.170406 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-469c9209-5da7-4348-b2e9-fc37bc2c7984 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:17.171930 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1057/2094] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:20:17 2022] DELETE /identity/v3/projects/6c912ecd1bda4a45b70176da39c19e1e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:17.176860 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490d03b-2d92-4047-adad-8f67038edc5f None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:17.177356 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490d03b-2d92-4047-adad-8f67038edc5f None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:17.177446 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490d03b-2d92-4047-adad-8f67038edc5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:17.197005 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-5490d03b-2d92-4047-adad-8f67038edc5f None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:17.198178 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-5490d03b-2d92-4047-adad-8f67038edc5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:17.259920 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1038/2095] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:17.268922 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5637c49-abe8-4f42-84d6-674d0d97b8a2 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:17.269211 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5637c49-abe8-4f42-84d6-674d0d97b8a2 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:17.269685 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5637c49-abe8-4f42-84d6-674d0d97b8a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:17.291073 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e5637c49-abe8-4f42-84d6-674d0d97b8a2 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:17.292290 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e5637c49-abe8-4f42-84d6-674d0d97b8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:17.301878 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1058/2096] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:17.306473 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490d03b-2d92-4047-adad-8f67038edc5f None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:17.316561 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490d03b-2d92-4047-adad-8f67038edc5f None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:17.321353 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601872eb-eb8a-42aa-b1dd-341ae8b1dfa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:17.321947 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601872eb-eb8a-42aa-b1dd-341ae8b1dfa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:17.322259 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601872eb-eb8a-42aa-b1dd-341ae8b1dfa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:17.322523 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601872eb-eb8a-42aa-b1dd-341ae8b1dfa1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:17.323173 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601872eb-eb8a-42aa-b1dd-341ae8b1dfa1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:20:17.323466 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601872eb-eb8a-42aa-b1dd-341ae8b1dfa1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:17.325111 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601872eb-eb8a-42aa-b1dd-341ae8b1dfa1 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:17.337073 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1039/2097] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:20:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:20:17.341589 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5637c49-abe8-4f42-84d6-674d0d97b8a2 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:17.351164 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5637c49-abe8-4f42-84d6-674d0d97b8a2 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:17.355932 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da354c8-2e79-45f2-812d-46b1d23b61e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:17.356528 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da354c8-2e79-45f2-812d-46b1d23b61e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:17.356800 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da354c8-2e79-45f2-812d-46b1d23b61e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:17.357063 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da354c8-2e79-45f2-812d-46b1d23b61e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:17.357694 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da354c8-2e79-45f2-812d-46b1d23b61e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:17.359981 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da354c8-2e79-45f2-812d-46b1d23b61e9 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:17.389052 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1059/2098] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:20:17 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:20:17.393584 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601872eb-eb8a-42aa-b1dd-341ae8b1dfa1 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:17.404359 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601872eb-eb8a-42aa-b1dd-341ae8b1dfa1 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:17.463773 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8762e390-d91e-4fb6-ab54-1959f2bbc881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:17.464381 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762e390-d91e-4fb6-ab54-1959f2bbc881 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:17.464681 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762e390-d91e-4fb6-ab54-1959f2bbc881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:17.464941 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762e390-d91e-4fb6-ab54-1959f2bbc881 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:17.465610 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8762e390-d91e-4fb6-ab54-1959f2bbc881 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:17.467561 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8762e390-d91e-4fb6-ab54-1959f2bbc881 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:17.490628 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1040/2099] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:20:17 2022] POST /identity/v3/users => generated 285 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:20:17.494884 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da354c8-2e79-45f2-812d-46b1d23b61e9 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:17.505665 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da354c8-2e79-45f2-812d-46b1d23b61e9 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:17.511045 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f6dd9-7027-49aa-8d94-3127997d1e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:17.511597 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f6dd9-7027-49aa-8d94-3127997d1e4c admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:17.511876 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f6dd9-7027-49aa-8d94-3127997d1e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:17.512144 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f6dd9-7027-49aa-8d94-3127997d1e4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:17.512828 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74f6dd9-7027-49aa-8d94-3127997d1e4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:17.514518 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74f6dd9-7027-49aa-8d94-3127997d1e4c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:17.528672 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1060/2100] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:20:17.532740 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8762e390-d91e-4fb6-ab54-1959f2bbc881 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:17.543625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8762e390-d91e-4fb6-ab54-1959f2bbc881 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:17.549114 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d0f12-29f1-47d4-87dc-310f14bfd93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:17.549730 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d0f12-29f1-47d4-87dc-310f14bfd93f admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:17.550103 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d0f12-29f1-47d4-87dc-310f14bfd93f admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:17.550364 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d0f12-29f1-47d4-87dc-310f14bfd93f admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:17.551005 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d0f12-29f1-47d4-87dc-310f14bfd93f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:17.552614 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d0f12-29f1-47d4-87dc-310f14bfd93f admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:17.564115 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1041/2101] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:20:17.568353 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f6dd9-7027-49aa-8d94-3127997d1e4c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:17.581820 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f6dd9-7027-49aa-8d94-3127997d1e4c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:17.590200 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc26b83-05aa-4d60-9f2f-5f0fbbbd76fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:17.590998 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc26b83-05aa-4d60-9f2f-5f0fbbbd76fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:17.591318 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc26b83-05aa-4d60-9f2f-5f0fbbbd76fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:17.591588 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc26b83-05aa-4d60-9f2f-5f0fbbbd76fb admin admin] PATH_INFO: `/v3/projects/7b6607d583b24a5fb69232e5b4bfcbfb/users/6e4d132c55244deb97e94a5063206d29/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:17.607439 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc26b83-05aa-4d60-9f2f-5f0fbbbd76fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b6607d583b24a5fb69232e5b4bfcbfb, user_id=6e4d132c55244deb97e94a5063206d29, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:17.610965 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc26b83-05aa-4d60-9f2f-5f0fbbbd76fb admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:17.625326 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1061/2102] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:20:17 2022] PUT /identity/v3/projects/7b6607d583b24a5fb69232e5b4bfcbfb/users/6e4d132c55244deb97e94a5063206d29/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:17.635792 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c46cb5c-8ef5-47b9-a305-1fd8cd3917b8 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:17.636075 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c46cb5c-8ef5-47b9-a305-1fd8cd3917b8 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:17.636341 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c46cb5c-8ef5-47b9-a305-1fd8cd3917b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:17.671651 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-6c46cb5c-8ef5-47b9-a305-1fd8cd3917b8 None None] MFA Rules not processed for user `6e4d132c55244deb97e94a5063206d29`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:17.672963 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-6c46cb5c-8ef5-47b9-a305-1fd8cd3917b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:17.726645 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:17.746401 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1042/2103] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:17 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:17.751144 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc26b83-05aa-4d60-9f2f-5f0fbbbd76fb admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:17.823445 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc26b83-05aa-4d60-9f2f-5f0fbbbd76fb admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:17.829478 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b38bf6-d4ce-47e5-881b-bb1d6dce9b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:17.830273 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b38bf6-d4ce-47e5-881b-bb1d6dce9b10 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:17.830551 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b38bf6-d4ce-47e5-881b-bb1d6dce9b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:17.830818 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b38bf6-d4ce-47e5-881b-bb1d6dce9b10 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:17.831433 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b38bf6-d4ce-47e5-881b-bb1d6dce9b10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:17.834186 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b38bf6-d4ce-47e5-881b-bb1d6dce9b10 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:17.864003 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1062/2104] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:20:17 2022] POST /identity/v3/projects => generated 367 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:20:17.868133 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c46cb5c-8ef5-47b9-a305-1fd8cd3917b8 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:17.879728 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c46cb5c-8ef5-47b9-a305-1fd8cd3917b8 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:17.940067 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18947864-f96e-4651-8a5a-607e43c97daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:17.940886 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18947864-f96e-4651-8a5a-607e43c97daa admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:17.941300 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18947864-f96e-4651-8a5a-607e43c97daa admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:17.941638 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18947864-f96e-4651-8a5a-607e43c97daa admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:17.942606 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18947864-f96e-4651-8a5a-607e43c97daa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:17.945638 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18947864-f96e-4651-8a5a-607e43c97daa admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:17.970726 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1043/2105] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:20:17 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:20:17.974942 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b38bf6-d4ce-47e5-881b-bb1d6dce9b10 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:17.985916 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b38bf6-d4ce-47e5-881b-bb1d6dce9b10 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:17.990782 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf576f87-d87a-4fd3-b484-b33e7fa0bfa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:17.991369 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf576f87-d87a-4fd3-b484-b33e7fa0bfa3 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:17.991639 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf576f87-d87a-4fd3-b484-b33e7fa0bfa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:17.991902 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf576f87-d87a-4fd3-b484-b33e7fa0bfa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:17.992533 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf576f87-d87a-4fd3-b484-b33e7fa0bfa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:17.994165 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf576f87-d87a-4fd3-b484-b33e7fa0bfa3 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:18.005004 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1063/2106] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:20:18.010101 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18947864-f96e-4651-8a5a-607e43c97daa admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:18.025453 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18947864-f96e-4651-8a5a-607e43c97daa admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:18.032839 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e5c76f-b75f-4383-b601-a80be575ef8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:18.033590 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e5c76f-b75f-4383-b601-a80be575ef8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:18.033891 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e5c76f-b75f-4383-b601-a80be575ef8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:18.034284 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e5c76f-b75f-4383-b601-a80be575ef8f admin admin] PATH_INFO: `/v3/projects/a6e4427f0da440ffacb4376cf81de350/users/376bc51622e245cca08162ab9e6c7d81/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:18.050795 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e5c76f-b75f-4383-b601-a80be575ef8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6e4427f0da440ffacb4376cf81de350, user_id=376bc51622e245cca08162ab9e6c7d81, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:18.053450 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e5c76f-b75f-4383-b601-a80be575ef8f admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:18.068444 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1044/2107] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:20:18 2022] PUT /identity/v3/projects/a6e4427f0da440ffacb4376cf81de350/users/376bc51622e245cca08162ab9e6c7d81/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:18.072574 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf576f87-d87a-4fd3-b484-b33e7fa0bfa3 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:18.083146 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf576f87-d87a-4fd3-b484-b33e7fa0bfa3 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:18.092451 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:18.145864 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91c391a-60d1-4edf-9c1c-4e1b1c3e6163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:18.146501 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c391a-60d1-4edf-9c1c-4e1b1c3e6163 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:18.146774 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c391a-60d1-4edf-9c1c-4e1b1c3e6163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:18.147035 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c391a-60d1-4edf-9c1c-4e1b1c3e6163 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:18.147815 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91c391a-60d1-4edf-9c1c-4e1b1c3e6163 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:18.149466 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91c391a-60d1-4edf-9c1c-4e1b1c3e6163 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:18.160438 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1064/2108] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:18 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:20:18.164880 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e5c76f-b75f-4383-b601-a80be575ef8f admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:18.175805 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e5c76f-b75f-4383-b601-a80be575ef8f admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:18.181085 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19fa628-0cf3-4aa0-8021-b33ddb425fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:18.181684 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19fa628-0cf3-4aa0-8021-b33ddb425fa2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:18.181997 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19fa628-0cf3-4aa0-8021-b33ddb425fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:18.182306 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19fa628-0cf3-4aa0-8021-b33ddb425fa2 admin admin] PATH_INFO: `/v3/projects/a6e4427f0da440ffacb4376cf81de350/users/376bc51622e245cca08162ab9e6c7d81/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:18.184372 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19fa628-0cf3-4aa0-8021-b33ddb425fa2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6e4427f0da440ffacb4376cf81de350, user_id=376bc51622e245cca08162ab9e6c7d81, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:18.187105 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19fa628-0cf3-4aa0-8021-b33ddb425fa2 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:18.197518 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1045/2109] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:20:18 2022] PUT /identity/v3/projects/a6e4427f0da440ffacb4376cf81de350/users/376bc51622e245cca08162ab9e6c7d81/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:18.207789 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf36d0-6541-446a-894c-9952d943b2f9 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:18.208052 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf36d0-6541-446a-894c-9952d943b2f9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:18.208342 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf36d0-6541-446a-894c-9952d943b2f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:18.241792 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-eebf36d0-6541-446a-894c-9952d943b2f9 None None] MFA Rules not processed for user `376bc51622e245cca08162ab9e6c7d81`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:18.243150 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-eebf36d0-6541-446a-894c-9952d943b2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:18.314048 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1065/2110] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:18 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:18.326235 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19fa628-0cf3-4aa0-8021-b33ddb425fa2 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:18.328254 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-f19fa628-0cf3-4aa0-8021-b33ddb425fa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:18.394315 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19fa628-0cf3-4aa0-8021-b33ddb425fa2 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:18.398581 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e31fb69-51f2-4da2-8cb2-9a46ea867cec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:18.399530 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e31fb69-51f2-4da2-8cb2-9a46ea867cec service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:18.399899 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e31fb69-51f2-4da2-8cb2-9a46ea867cec service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:18.400166 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e31fb69-51f2-4da2-8cb2-9a46ea867cec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:18.414887 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e31fb69-51f2-4da2-8cb2-9a46ea867cec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:18.417130 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e31fb69-51f2-4da2-8cb2-9a46ea867cec service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:18.426740 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1046/2111] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:20:18 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:20:18.474257 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebf36d0-6541-446a-894c-9952d943b2f9 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:18.475764 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-eebf36d0-6541-446a-894c-9952d943b2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:18.491399 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebf36d0-6541-446a-894c-9952d943b2f9 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:18.495375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b23906-38d6-422d-afeb-42e0394aca7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:18.495973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b23906-38d6-422d-afeb-42e0394aca7e service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:18.496299 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b23906-38d6-422d-afeb-42e0394aca7e service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:18.496482 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b23906-38d6-422d-afeb-42e0394aca7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:18.565124 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b23906-38d6-422d-afeb-42e0394aca7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:18.566722 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b23906-38d6-422d-afeb-42e0394aca7e service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:18.576076 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1066/2112] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:20:18 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:20:21.971843 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e31fb69-51f2-4da2-8cb2-9a46ea867cec service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:21.973241 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b23906-38d6-422d-afeb-42e0394aca7e service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:21.974717 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-06b23906-38d6-422d-afeb-42e0394aca7e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:22.016445 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:22.024040 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:22.040469 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e31fb69-51f2-4da2-8cb2-9a46ea867cec service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:22.045139 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a0e3e-256a-4f3a-9674-0138d8ae1b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:22.045724 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a0e3e-256a-4f3a-9674-0138d8ae1b24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:22.046020 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a0e3e-256a-4f3a-9674-0138d8ae1b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:22.046303 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a0e3e-256a-4f3a-9674-0138d8ae1b24 admin admin] PATH_INFO: `/v3/users/6e4d132c55244deb97e94a5063206d29` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:22.047561 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14a0e3e-256a-4f3a-9674-0138d8ae1b24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e4d132c55244deb97e94a5063206d29)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:22.049498 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14a0e3e-256a-4f3a-9674-0138d8ae1b24 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:22.053389 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b23906-38d6-422d-afeb-42e0394aca7e service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:22.058224 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547fd36c-f353-4034-9a7b-bf4a7b672376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:22.058856 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547fd36c-f353-4034-9a7b-bf4a7b672376 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:22.059193 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547fd36c-f353-4034-9a7b-bf4a7b672376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:22.059466 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547fd36c-f353-4034-9a7b-bf4a7b672376 admin admin] PATH_INFO: `/v3/users/ba4f2506c8e540bd84e584c7d1bc838d` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:22.060610 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547fd36c-f353-4034-9a7b-bf4a7b672376 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba4f2506c8e540bd84e584c7d1bc838d)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:22.062552 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547fd36c-f353-4034-9a7b-bf4a7b672376 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:22.101225 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c14a0e3e-256a-4f3a-9674-0138d8ae1b24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e4d132c55244deb97e94a5063206d29'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:22.111151 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c14a0e3e-256a-4f3a-9674-0138d8ae1b24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e4d132c55244deb97e94a5063206d29'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:22.111809 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c14a0e3e-256a-4f3a-9674-0138d8ae1b24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e4d132c55244deb97e94a5063206d29'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:22.113171 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-547fd36c-f353-4034-9a7b-bf4a7b672376 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba4f2506c8e540bd84e584c7d1bc838d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:22.120576 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c14a0e3e-256a-4f3a-9674-0138d8ae1b24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e4d132c55244deb97e94a5063206d29'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:22.120997 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-547fd36c-f353-4034-9a7b-bf4a7b672376 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba4f2506c8e540bd84e584c7d1bc838d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:22.121181 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c14a0e3e-256a-4f3a-9674-0138d8ae1b24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e4d132c55244deb97e94a5063206d29'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:22.129140 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-547fd36c-f353-4034-9a7b-bf4a7b672376 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba4f2506c8e540bd84e584c7d1bc838d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:22.129892 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-547fd36c-f353-4034-9a7b-bf4a7b672376 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba4f2506c8e540bd84e584c7d1bc838d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:22.141193 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1047/2113] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:20:21 2022] DELETE /identity/v3/users/6e4d132c55244deb97e94a5063206d29 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:22.142100 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-547fd36c-f353-4034-9a7b-bf4a7b672376 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba4f2506c8e540bd84e584c7d1bc838d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:22.145440 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31d7d6-c032-4f74-b6ef-42741f459c26 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:22.145714 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31d7d6-c032-4f74-b6ef-42741f459c26 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:22.145981 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31d7d6-c032-4f74-b6ef-42741f459c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:22.148462 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1067/2114] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:20:21 2022] DELETE /identity/v3/users/ba4f2506c8e540bd84e584c7d1bc838d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:22.153177 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eed744-d868-4f6d-903a-68193dac5459 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:22.153958 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eed744-d868-4f6d-903a-68193dac5459 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:22.154430 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eed744-d868-4f6d-903a-68193dac5459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:22.167406 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-be31d7d6-c032-4f74-b6ef-42741f459c26 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:22.168764 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-be31d7d6-c032-4f74-b6ef-42741f459c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:22.175708 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f7eed744-d868-4f6d-903a-68193dac5459 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:22.176867 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f7eed744-d868-4f6d-903a-68193dac5459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:22.230628 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1048/2115] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:22.239444 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1068/2116] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:20:22.242594 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be31d7d6-c032-4f74-b6ef-42741f459c26 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:22.242594 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-be31d7d6-c032-4f74-b6ef-42741f459c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:22.248232 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eed744-d868-4f6d-903a-68193dac5459 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:22.249619 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f7eed744-d868-4f6d-903a-68193dac5459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:22.306509 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be31d7d6-c032-4f74-b6ef-42741f459c26 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:22.307766 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eed744-d868-4f6d-903a-68193dac5459 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:22.310444 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1108ff4-4b76-4425-a007-69cd20b0af7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:22.311063 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1108ff4-4b76-4425-a007-69cd20b0af7f service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:22.311340 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1108ff4-4b76-4425-a007-69cd20b0af7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:22.311439 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9310ecbb-119c-4405-bd75-56dedcc4d6dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:22.311643 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1108ff4-4b76-4425-a007-69cd20b0af7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:22.311963 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9310ecbb-119c-4405-bd75-56dedcc4d6dc service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:22.312235 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9310ecbb-119c-4405-bd75-56dedcc4d6dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:22.312499 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9310ecbb-119c-4405-bd75-56dedcc4d6dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:22.321246 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1108ff4-4b76-4425-a007-69cd20b0af7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:22.322728 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1108ff4-4b76-4425-a007-69cd20b0af7f service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:22.327716 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:22.330794 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9310ecbb-119c-4405-bd75-56dedcc4d6dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:22.332700 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1049/2117] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:20:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:20:22.333370 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9310ecbb-119c-4405-bd75-56dedcc4d6dc service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:22.344507 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1069/2118] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:20:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 05:20:22.846739 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1108ff4-4b76-4425-a007-69cd20b0af7f service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:22.848161 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c1108ff4-4b76-4425-a007-69cd20b0af7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:22.853873 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9310ecbb-119c-4405-bd75-56dedcc4d6dc service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:22.855398 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9310ecbb-119c-4405-bd75-56dedcc4d6dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:22.864211 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1108ff4-4b76-4425-a007-69cd20b0af7f service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:22.876403 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9588a9e-7452-4b21-ae39-c286b36d0867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:22.877058 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9588a9e-7452-4b21-ae39-c286b36d0867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:22.877337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9588a9e-7452-4b21-ae39-c286b36d0867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:22.878087 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9588a9e-7452-4b21-ae39-c286b36d0867 admin admin] PATH_INFO: `/v3/projects/7b6607d583b24a5fb69232e5b4bfcbfb` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:22.879103 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9588a9e-7452-4b21-ae39-c286b36d0867 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b6607d583b24a5fb69232e5b4bfcbfb)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:22.880903 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9588a9e-7452-4b21-ae39-c286b36d0867 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:22.882161 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9310ecbb-119c-4405-bd75-56dedcc4d6dc service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:22.883825 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:22.887856 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c95cfe-ee23-476f-ae0a-5c31798fdf32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:22.888590 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c95cfe-ee23-476f-ae0a-5c31798fdf32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:22.888937 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c95cfe-ee23-476f-ae0a-5c31798fdf32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:22.889282 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c95cfe-ee23-476f-ae0a-5c31798fdf32 admin admin] PATH_INFO: `/v3/projects/8b3cfc1ea7e74cd3b92f51e8c56b8ed8` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:22.891088 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c95cfe-ee23-476f-ae0a-5c31798fdf32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b3cfc1ea7e74cd3b92f51e8c56b8ed8)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:22.893517 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c95cfe-ee23-476f-ae0a-5c31798fdf32 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:22.949333 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f9588a9e-7452-4b21-ae39-c286b36d0867 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b6607d583b24a5fb69232e5b4bfcbfb', 'request_id': 'req-f9588a9e-7452-4b21-ae39-c286b36d0867'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:22.956765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f9588a9e-7452-4b21-ae39-c286b36d0867 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b6607d583b24a5fb69232e5b4bfcbfb', 'request_id': 'req-f9588a9e-7452-4b21-ae39-c286b36d0867'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:22.964605 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f9588a9e-7452-4b21-ae39-c286b36d0867 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b6607d583b24a5fb69232e5b4bfcbfb', 'request_id': 'req-f9588a9e-7452-4b21-ae39-c286b36d0867'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:22.964754 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-07c95cfe-ee23-476f-ae0a-5c31798fdf32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b3cfc1ea7e74cd3b92f51e8c56b8ed8', 'request_id': 'req-07c95cfe-ee23-476f-ae0a-5c31798fdf32'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:22.972685 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-07c95cfe-ee23-476f-ae0a-5c31798fdf32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b3cfc1ea7e74cd3b92f51e8c56b8ed8', 'request_id': 'req-07c95cfe-ee23-476f-ae0a-5c31798fdf32'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:22.979235 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f9588a9e-7452-4b21-ae39-c286b36d0867 admin admin] The token cache is being invalidate because project 7b6607d583b24a5fb69232e5b4bfcbfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:20:22.979993 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f9588a9e-7452-4b21-ae39-c286b36d0867 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:22.980364 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-f9588a9e-7452-4b21-ae39-c286b36d0867 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:22.982052 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1050/2119] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:20:22 2022] DELETE /identity/v3/projects/7b6607d583b24a5fb69232e5b4bfcbfb => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:22.986283 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9588a9e-7452-4b21-ae39-c286b36d0867 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:22.987365 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-07c95cfe-ee23-476f-ae0a-5c31798fdf32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b3cfc1ea7e74cd3b92f51e8c56b8ed8', 'request_id': 'req-07c95cfe-ee23-476f-ae0a-5c31798fdf32'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:22.988469 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-f9588a9e-7452-4b21-ae39-c286b36d0867 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:23.002676 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-07c95cfe-ee23-476f-ae0a-5c31798fdf32 admin admin] The token cache is being invalidate because project 8b3cfc1ea7e74cd3b92f51e8c56b8ed8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:20:23.003087 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-07c95cfe-ee23-476f-ae0a-5c31798fdf32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:23.003827 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-07c95cfe-ee23-476f-ae0a-5c31798fdf32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:23.008838 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1070/2120] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:20:22 2022] DELETE /identity/v3/projects/8b3cfc1ea7e74cd3b92f51e8c56b8ed8 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:23.010082 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c95cfe-ee23-476f-ae0a-5c31798fdf32 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:23.011454 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-07c95cfe-ee23-476f-ae0a-5c31798fdf32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:23.065361 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9588a9e-7452-4b21-ae39-c286b36d0867 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:23.085898 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c95cfe-ee23-476f-ae0a-5c31798fdf32 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:23.090914 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396722af-2a73-4a76-bda2-c3a5ee6ba2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:23.091540 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396722af-2a73-4a76-bda2-c3a5ee6ba2c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:23.092050 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396722af-2a73-4a76-bda2-c3a5ee6ba2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:23.092179 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396722af-2a73-4a76-bda2-c3a5ee6ba2c6 admin admin] PATH_INFO: `/v3/users/4726edc6dd144d719a46572fa60a6d6e` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:23.093260 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396722af-2a73-4a76-bda2-c3a5ee6ba2c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4726edc6dd144d719a46572fa60a6d6e)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:23.095304 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396722af-2a73-4a76-bda2-c3a5ee6ba2c6 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:23.136015 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c909e6cf-949f-41a9-b2c0-6651796a3c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:23.136717 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909e6cf-949f-41a9-b2c0-6651796a3c70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:23.137026 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909e6cf-949f-41a9-b2c0-6651796a3c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:23.137345 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909e6cf-949f-41a9-b2c0-6651796a3c70 admin admin] PATH_INFO: `/v3/users/376bc51622e245cca08162ab9e6c7d81` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:23.138780 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c909e6cf-949f-41a9-b2c0-6651796a3c70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=376bc51622e245cca08162ab9e6c7d81)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:23.140737 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c909e6cf-949f-41a9-b2c0-6651796a3c70 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:23.147883 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-396722af-2a73-4a76-bda2-c3a5ee6ba2c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4726edc6dd144d719a46572fa60a6d6e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:23.155258 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-396722af-2a73-4a76-bda2-c3a5ee6ba2c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4726edc6dd144d719a46572fa60a6d6e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:23.163496 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-396722af-2a73-4a76-bda2-c3a5ee6ba2c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4726edc6dd144d719a46572fa60a6d6e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:23.164170 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-396722af-2a73-4a76-bda2-c3a5ee6ba2c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4726edc6dd144d719a46572fa60a6d6e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:23.176925 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-396722af-2a73-4a76-bda2-c3a5ee6ba2c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4726edc6dd144d719a46572fa60a6d6e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:23.178389 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:23.183682 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:23.184058 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1071/2121] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:20:23 2022] DELETE /identity/v3/users/4726edc6dd144d719a46572fa60a6d6e => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:20:23.185148 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396722af-2a73-4a76-bda2-c3a5ee6ba2c6 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:23.186562 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-396722af-2a73-4a76-bda2-c3a5ee6ba2c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:23.189541 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c909e6cf-949f-41a9-b2c0-6651796a3c70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '376bc51622e245cca08162ab9e6c7d81'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:23.199734 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c909e6cf-949f-41a9-b2c0-6651796a3c70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '376bc51622e245cca08162ab9e6c7d81'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:23.200444 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c909e6cf-949f-41a9-b2c0-6651796a3c70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '376bc51622e245cca08162ab9e6c7d81'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:23.213796 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c909e6cf-949f-41a9-b2c0-6651796a3c70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '376bc51622e245cca08162ab9e6c7d81'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:23.214884 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c909e6cf-949f-41a9-b2c0-6651796a3c70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '376bc51622e245cca08162ab9e6c7d81'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:23.237278 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1051/2122] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:20:22 2022] DELETE /identity/v3/users/376bc51622e245cca08162ab9e6c7d81 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:23.295756 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396722af-2a73-4a76-bda2-c3a5ee6ba2c6 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:23.371969 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8daecd-a60d-42b5-9746-206097ba8777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:23.372732 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8daecd-a60d-42b5-9746-206097ba8777 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:23.373132 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8daecd-a60d-42b5-9746-206097ba8777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:23.373473 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8daecd-a60d-42b5-9746-206097ba8777 admin admin] PATH_INFO: `/v3/users/3bb39ec582b9410eab580e4e07871f07` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:23.374771 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8daecd-a60d-42b5-9746-206097ba8777 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bb39ec582b9410eab580e4e07871f07)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:23.376880 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8daecd-a60d-42b5-9746-206097ba8777 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:23.434145 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9c8daecd-a60d-42b5-9746-206097ba8777 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bb39ec582b9410eab580e4e07871f07'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:23.444516 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9c8daecd-a60d-42b5-9746-206097ba8777 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bb39ec582b9410eab580e4e07871f07'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:23.453895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9c8daecd-a60d-42b5-9746-206097ba8777 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bb39ec582b9410eab580e4e07871f07'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:23.454751 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9c8daecd-a60d-42b5-9746-206097ba8777 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bb39ec582b9410eab580e4e07871f07'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:23.469797 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9c8daecd-a60d-42b5-9746-206097ba8777 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bb39ec582b9410eab580e4e07871f07'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:23.480359 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1072/2123] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:20:23 2022] DELETE /identity/v3/users/3bb39ec582b9410eab580e4e07871f07 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:23.485489 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea974199-00e2-41a0-bf25-724e1c1e8bb8 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:23.485938 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea974199-00e2-41a0-bf25-724e1c1e8bb8 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:23.486244 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea974199-00e2-41a0-bf25-724e1c1e8bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:23.511588 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-ea974199-00e2-41a0-bf25-724e1c1e8bb8 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:23.512828 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-ea974199-00e2-41a0-bf25-724e1c1e8bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:23.522577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8daecd-a60d-42b5-9746-206097ba8777 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:23.531831 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9c8daecd-a60d-42b5-9746-206097ba8777 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:23.570704 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:23.603023 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1052/2124] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:20:23.603023 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea974199-00e2-41a0-bf25-724e1c1e8bb8 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:23.603023 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-ea974199-00e2-41a0-bf25-724e1c1e8bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:23.616678 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea974199-00e2-41a0-bf25-724e1c1e8bb8 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:23.622980 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc25f5f9-45c8-423e-87a5-b866cf7330d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:23.624330 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc25f5f9-45c8-423e-87a5-b866cf7330d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:23.624703 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc25f5f9-45c8-423e-87a5-b866cf7330d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:23.625215 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc25f5f9-45c8-423e-87a5-b866cf7330d4 admin admin] PATH_INFO: `/v3/projects/a6e4427f0da440ffacb4376cf81de350` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:23.625861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8daecd-a60d-42b5-9746-206097ba8777 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:23.626818 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc25f5f9-45c8-423e-87a5-b866cf7330d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6e4427f0da440ffacb4376cf81de350)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:23.629110 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc25f5f9-45c8-423e-87a5-b866cf7330d4 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:23.631078 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78944625-0bcf-4bc1-bd00-7efd35f5f19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:23.631725 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78944625-0bcf-4bc1-bd00-7efd35f5f19c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:23.632003 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78944625-0bcf-4bc1-bd00-7efd35f5f19c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:23.632436 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78944625-0bcf-4bc1-bd00-7efd35f5f19c admin admin] PATH_INFO: `/v3/projects/85edb18dc5b8448ca3ade80eaa87d45d` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:23.633663 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78944625-0bcf-4bc1-bd00-7efd35f5f19c admin admin] RBAC: Authorizing `identity:delete_project(project_id=85edb18dc5b8448ca3ade80eaa87d45d)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:23.635897 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78944625-0bcf-4bc1-bd00-7efd35f5f19c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:23.691569 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-78944625-0bcf-4bc1-bd00-7efd35f5f19c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85edb18dc5b8448ca3ade80eaa87d45d', 'request_id': 'req-78944625-0bcf-4bc1-bd00-7efd35f5f19c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:23.692145 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-cc25f5f9-45c8-423e-87a5-b866cf7330d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6e4427f0da440ffacb4376cf81de350', 'request_id': 'req-cc25f5f9-45c8-423e-87a5-b866cf7330d4'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:23.698069 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-78944625-0bcf-4bc1-bd00-7efd35f5f19c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85edb18dc5b8448ca3ade80eaa87d45d', 'request_id': 'req-78944625-0bcf-4bc1-bd00-7efd35f5f19c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:23.698375 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-cc25f5f9-45c8-423e-87a5-b866cf7330d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6e4427f0da440ffacb4376cf81de350', 'request_id': 'req-cc25f5f9-45c8-423e-87a5-b866cf7330d4'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:23.706865 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-cc25f5f9-45c8-423e-87a5-b866cf7330d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6e4427f0da440ffacb4376cf81de350', 'request_id': 'req-cc25f5f9-45c8-423e-87a5-b866cf7330d4'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:23.707654 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-78944625-0bcf-4bc1-bd00-7efd35f5f19c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85edb18dc5b8448ca3ade80eaa87d45d', 'request_id': 'req-78944625-0bcf-4bc1-bd00-7efd35f5f19c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:23.721638 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-78944625-0bcf-4bc1-bd00-7efd35f5f19c admin admin] The token cache is being invalidate because project 85edb18dc5b8448ca3ade80eaa87d45d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:20:23.721853 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-cc25f5f9-45c8-423e-87a5-b866cf7330d4 admin admin] The token cache is being invalidate because project a6e4427f0da440ffacb4376cf81de350 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:20:23.721955 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-78944625-0bcf-4bc1-bd00-7efd35f5f19c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:23.722241 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-cc25f5f9-45c8-423e-87a5-b866cf7330d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:23.722624 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-78944625-0bcf-4bc1-bd00-7efd35f5f19c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:23.722850 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-cc25f5f9-45c8-423e-87a5-b866cf7330d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:23.724246 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1073/2125] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:20:23 2022] DELETE /identity/v3/projects/85edb18dc5b8448ca3ade80eaa87d45d => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:23.728141 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1053/2126] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:20:23 2022] DELETE /identity/v3/projects/a6e4427f0da440ffacb4376cf81de350 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:23.728299 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78944625-0bcf-4bc1-bd00-7efd35f5f19c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:23.729576 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30016407-5f08-472d-a553-74142dbbc970 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:23.729857 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30016407-5f08-472d-a553-74142dbbc970 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:23.730407 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30016407-5f08-472d-a553-74142dbbc970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:23.731060 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-78944625-0bcf-4bc1-bd00-7efd35f5f19c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:23.753871 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-30016407-5f08-472d-a553-74142dbbc970 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:23.755288 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-30016407-5f08-472d-a553-74142dbbc970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:23.790679 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78944625-0bcf-4bc1-bd00-7efd35f5f19c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:23.795633 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:23.834150 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1054/2127] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:20:23.837058 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fab095-7b01-4f39-ad31-d015dfbd5351 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:23.837725 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fab095-7b01-4f39-ad31-d015dfbd5351 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:23.838184 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fab095-7b01-4f39-ad31-d015dfbd5351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:23.845915 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f2d98-b006-43ab-9b4f-b0d11f7b172b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:23.850932 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f2d98-b006-43ab-9b4f-b0d11f7b172b service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:23.851291 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f2d98-b006-43ab-9b4f-b0d11f7b172b service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:23.851628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f2d98-b006-43ab-9b4f-b0d11f7b172b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:23.858989 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7f4f2d98-b006-43ab-9b4f-b0d11f7b172b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:23.861181 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-53fab095-7b01-4f39-ad31-d015dfbd5351 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:23.862652 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-53fab095-7b01-4f39-ad31-d015dfbd5351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:23.870746 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1055/2128] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:23.871899 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c14fa-762a-4249-a258-0f0c6f7da569 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:23.872175 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c14fa-762a-4249-a258-0f0c6f7da569 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:23.872674 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c14fa-762a-4249-a258-0f0c6f7da569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:23.873039 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4f2d98-b006-43ab-9b4f-b0d11f7b172b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:23.874562 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4f2d98-b006-43ab-9b4f-b0d11f7b172b service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:23.885358 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1074/2129] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:20:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:20:23.886943 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab2d91-68fd-4724-adbf-28f8b04808b9 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:23.887367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab2d91-68fd-4724-adbf-28f8b04808b9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:23.887739 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab2d91-68fd-4724-adbf-28f8b04808b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:23.904097 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-4c5c14fa-762a-4249-a258-0f0c6f7da569 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:23.905340 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-4c5c14fa-762a-4249-a258-0f0c6f7da569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:23.910429 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-68ab2d91-68fd-4724-adbf-28f8b04808b9 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:23.912287 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-68ab2d91-68fd-4724-adbf-28f8b04808b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:23.914899 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1056/2130] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:23.921991 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5c14fa-762a-4249-a258-0f0c6f7da569 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:23.930063 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1075/2131] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:23.932758 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ab2d91-68fd-4724-adbf-28f8b04808b9 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:23.934223 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5c14fa-762a-4249-a258-0f0c6f7da569 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:23.945017 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ab2d91-68fd-4724-adbf-28f8b04808b9 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:23.947364 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211985f-f868-4a49-9e47-3edc192903d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:23.948131 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211985f-f868-4a49-9e47-3edc192903d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:23.948411 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211985f-f868-4a49-9e47-3edc192903d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:23.948680 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211985f-f868-4a49-9e47-3edc192903d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:23.949344 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b211985f-f868-4a49-9e47-3edc192903d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:20:23.949871 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b211985f-f868-4a49-9e47-3edc192903d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:23.951100 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea24717d-1a44-44e0-b9c3-e957988228fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:23.951540 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b211985f-f868-4a49-9e47-3edc192903d7 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:23.952119 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea24717d-1a44-44e0-b9c3-e957988228fe admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:23.952570 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea24717d-1a44-44e0-b9c3-e957988228fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:23.953012 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea24717d-1a44-44e0-b9c3-e957988228fe admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:23.954309 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea24717d-1a44-44e0-b9c3-e957988228fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:20:23.954677 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea24717d-1a44-44e0-b9c3-e957988228fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:23.957146 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea24717d-1a44-44e0-b9c3-e957988228fe admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:23.959241 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:23.970918 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1057/2132] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:20:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:20:23.976559 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211985f-f868-4a49-9e47-3edc192903d7 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:23.982343 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1076/2133] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:20:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:20:23.988937 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea24717d-1a44-44e0-b9c3-e957988228fe admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:23.999060 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211985f-f868-4a49-9e47-3edc192903d7 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:24.005143 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e994f25d-b21c-4a8b-a79e-0aca7124553f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:24.006357 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e994f25d-b21c-4a8b-a79e-0aca7124553f admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:24.006648 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e994f25d-b21c-4a8b-a79e-0aca7124553f admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:24.006924 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e994f25d-b21c-4a8b-a79e-0aca7124553f admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:24.007574 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e994f25d-b21c-4a8b-a79e-0aca7124553f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:24.010052 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e994f25d-b21c-4a8b-a79e-0aca7124553f admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:24.011496 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea24717d-1a44-44e0-b9c3-e957988228fe admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:24.016339 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a9a7b-25cb-4bb9-b46c-51173640f2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:24.016965 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a9a7b-25cb-4bb9-b46c-51173640f2a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:24.017256 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a9a7b-25cb-4bb9-b46c-51173640f2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:24.017486 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a9a7b-25cb-4bb9-b46c-51173640f2a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:24.018163 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a9a7b-25cb-4bb9-b46c-51173640f2a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:24.020069 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a9a7b-25cb-4bb9-b46c-51173640f2a2 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:24.047518 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1058/2134] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:20:23 2022] POST /identity/v3/projects => generated 381 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:20:24.051559 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e994f25d-b21c-4a8b-a79e-0aca7124553f admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:24.058460 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1077/2135] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:20:23 2022] POST /identity/v3/projects => generated 371 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:20:24.063417 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a9a7b-25cb-4bb9-b46c-51173640f2a2 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:24.130594 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e994f25d-b21c-4a8b-a79e-0aca7124553f admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:24.150147 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a9a7b-25cb-4bb9-b46c-51173640f2a2 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:24.155566 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a052737c-b4d5-4791-a8b4-786efc0c7c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:24.156389 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a052737c-b4d5-4791-a8b4-786efc0c7c60 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:24.156956 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a052737c-b4d5-4791-a8b4-786efc0c7c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:24.157411 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a052737c-b4d5-4791-a8b4-786efc0c7c60 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:24.158143 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a052737c-b4d5-4791-a8b4-786efc0c7c60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:24.160085 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a052737c-b4d5-4791-a8b4-786efc0c7c60 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:24.185091 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1078/2136] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:20:24 2022] POST /identity/v3/users => generated 286 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:20:24.188950 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a052737c-b4d5-4791-a8b4-786efc0c7c60 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:24.197232 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45286dfe-b254-4f74-af4d-58e35900eb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:24.198591 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45286dfe-b254-4f74-af4d-58e35900eb61 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:24.198877 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45286dfe-b254-4f74-af4d-58e35900eb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:24.199278 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45286dfe-b254-4f74-af4d-58e35900eb61 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:24.200072 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45286dfe-b254-4f74-af4d-58e35900eb61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:24.200197 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a052737c-b4d5-4791-a8b4-786efc0c7c60 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:24.202375 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45286dfe-b254-4f74-af4d-58e35900eb61 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:24.205662 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac8ecf-a607-4546-a773-3c85fabc47cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:24.206071 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac8ecf-a607-4546-a773-3c85fabc47cc admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:24.206347 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac8ecf-a607-4546-a773-3c85fabc47cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:24.206807 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac8ecf-a607-4546-a773-3c85fabc47cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:24.207519 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bac8ecf-a607-4546-a773-3c85fabc47cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:24.209358 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bac8ecf-a607-4546-a773-3c85fabc47cc admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:24.223035 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1079/2137] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 05:20:24.226903 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac8ecf-a607-4546-a773-3c85fabc47cc admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:24.227851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1059/2138] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:20:24 2022] POST /identity/v3/users => generated 291 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:20:24.234434 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45286dfe-b254-4f74-af4d-58e35900eb61 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:24.241457 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac8ecf-a607-4546-a773-3c85fabc47cc admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:24.248179 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aef069a-b9a3-4490-8214-8334a4b5e2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:24.249169 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45286dfe-b254-4f74-af4d-58e35900eb61 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:24.249404 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef069a-b9a3-4490-8214-8334a4b5e2be admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:24.249958 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef069a-b9a3-4490-8214-8334a4b5e2be admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:24.250530 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef069a-b9a3-4490-8214-8334a4b5e2be admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:24.251434 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aef069a-b9a3-4490-8214-8334a4b5e2be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:24.253473 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aef069a-b9a3-4490-8214-8334a4b5e2be admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:24.254032 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d72c23-4fc2-4599-9dad-6d4df007b624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:24.254651 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d72c23-4fc2-4599-9dad-6d4df007b624 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:24.254929 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d72c23-4fc2-4599-9dad-6d4df007b624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:24.255199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d72c23-4fc2-4599-9dad-6d4df007b624 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:24.255985 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d72c23-4fc2-4599-9dad-6d4df007b624 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:24.257607 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d72c23-4fc2-4599-9dad-6d4df007b624 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:24.271532 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1080/2139] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:24 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 05:20:24.271768 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1060/2140] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 05:20:24.274665 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aef069a-b9a3-4490-8214-8334a4b5e2be admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:24.274829 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d72c23-4fc2-4599-9dad-6d4df007b624 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:24.292269 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aef069a-b9a3-4490-8214-8334a4b5e2be admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:24.293487 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d72c23-4fc2-4599-9dad-6d4df007b624 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:24.297561 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d24acb-703f-4b12-96ed-d64730848082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:24.298366 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d24acb-703f-4b12-96ed-d64730848082 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:24.298744 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d24acb-703f-4b12-96ed-d64730848082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:24.299113 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d24acb-703f-4b12-96ed-d64730848082 admin admin] PATH_INFO: `/v3/projects/04da0465aa794d208d6dd9e2aab7d704/users/9d7f48e1763e4ff3b77ddb90e65045b0/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:24.299230 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf3229a-84d7-4288-80c4-a5ff60b98db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:24.299996 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3229a-84d7-4288-80c4-a5ff60b98db9 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:24.300112 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3229a-84d7-4288-80c4-a5ff60b98db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:24.300294 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3229a-84d7-4288-80c4-a5ff60b98db9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:24.300941 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf3229a-84d7-4288-80c4-a5ff60b98db9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:24.303221 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf3229a-84d7-4288-80c4-a5ff60b98db9 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:24.315545 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1061/2141] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:24 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:20:24.318089 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d24acb-703f-4b12-96ed-d64730848082 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04da0465aa794d208d6dd9e2aab7d704, user_id=9d7f48e1763e4ff3b77ddb90e65045b0, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:24.320808 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d24acb-703f-4b12-96ed-d64730848082 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:24.322124 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf3229a-84d7-4288-80c4-a5ff60b98db9 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:24.338753 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf3229a-84d7-4288-80c4-a5ff60b98db9 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:24.343657 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1081/2142] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:20:24 2022] PUT /identity/v3/projects/04da0465aa794d208d6dd9e2aab7d704/users/9d7f48e1763e4ff3b77ddb90e65045b0/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:24.344956 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:24.346291 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aafc0b-e7d0-43ef-8d15-4a63fcb5237a None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:24.346809 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aafc0b-e7d0-43ef-8d15-4a63fcb5237a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:24.347093 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aafc0b-e7d0-43ef-8d15-4a63fcb5237a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:24.367770 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-c6aafc0b-e7d0-43ef-8d15-4a63fcb5237a None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:24.369198 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c6aafc0b-e7d0-43ef-8d15-4a63fcb5237a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:24.420058 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad6c45a-5bce-4954-9127-d6637081037a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:24.420728 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad6c45a-5bce-4954-9127-d6637081037a admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:24.421002 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad6c45a-5bce-4954-9127-d6637081037a admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:24.421273 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad6c45a-5bce-4954-9127-d6637081037a admin admin] PATH_INFO: `/v3/projects/c1a8a60fc87643d4ae48b72a45191de3/users/9dd8e02e25c1461f8aee21589271d4b8/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:24.442495 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad6c45a-5bce-4954-9127-d6637081037a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1a8a60fc87643d4ae48b72a45191de3, user_id=9dd8e02e25c1461f8aee21589271d4b8, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:24.442816 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1082/2143] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:24.448771 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad6c45a-5bce-4954-9127-d6637081037a admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:24.455304 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aafc0b-e7d0-43ef-8d15-4a63fcb5237a None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:24.457784 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c6aafc0b-e7d0-43ef-8d15-4a63fcb5237a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:24.473899 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1062/2144] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:20:24 2022] PUT /identity/v3/projects/c1a8a60fc87643d4ae48b72a45191de3/users/9dd8e02e25c1461f8aee21589271d4b8/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:24.476025 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee612db-c4f2-46a6-bd96-83a9e3619293 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:24.476474 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee612db-c4f2-46a6-bd96-83a9e3619293 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:24.476746 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee612db-c4f2-46a6-bd96-83a9e3619293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:24.500159 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:24.509053 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-6ee612db-c4f2-46a6-bd96-83a9e3619293 None None] MFA Rules not processed for user `9dd8e02e25c1461f8aee21589271d4b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:24.510328 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-6ee612db-c4f2-46a6-bd96-83a9e3619293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:24.559811 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aafc0b-e7d0-43ef-8d15-4a63fcb5237a None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:24.588972 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1063/2145] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:20:24.592637 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee612db-c4f2-46a6-bd96-83a9e3619293 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:24.593315 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-6ee612db-c4f2-46a6-bd96-83a9e3619293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:24.620058 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53bf4f9-fe13-4256-bff6-156fa0a294da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:24.620671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53bf4f9-fe13-4256-bff6-156fa0a294da service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:24.620947 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53bf4f9-fe13-4256-bff6-156fa0a294da service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:24.622307 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53bf4f9-fe13-4256-bff6-156fa0a294da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:24.698848 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee612db-c4f2-46a6-bd96-83a9e3619293 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:24.706901 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e73fca-dc97-45f4-9414-9db183feba47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:24.707978 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e73fca-dc97-45f4-9414-9db183feba47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:24.708899 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e73fca-dc97-45f4-9414-9db183feba47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:24.709575 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e73fca-dc97-45f4-9414-9db183feba47 admin admin] PATH_INFO: `/v3/projects/046429fd547f41ae90b5354dfd8b075b` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:24.711018 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e73fca-dc97-45f4-9414-9db183feba47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=046429fd547f41ae90b5354dfd8b075b)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:24.712898 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e73fca-dc97-45f4-9414-9db183feba47 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:24.714234 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53bf4f9-fe13-4256-bff6-156fa0a294da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:24.715993 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53bf4f9-fe13-4256-bff6-156fa0a294da service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:24.728438 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1083/2146] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:20:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 05:20:24.729575 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53bf4f9-fe13-4256-bff6-156fa0a294da service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:24.740923 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53bf4f9-fe13-4256-bff6-156fa0a294da service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:24.746285 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47926ac0-3a2d-4551-8d98-dfd6da71fa6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:24.747202 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47926ac0-3a2d-4551-8d98-dfd6da71fa6c admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:24.747692 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47926ac0-3a2d-4551-8d98-dfd6da71fa6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:24.748171 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47926ac0-3a2d-4551-8d98-dfd6da71fa6c admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:24.749226 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47926ac0-3a2d-4551-8d98-dfd6da71fa6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:24.751500 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47926ac0-3a2d-4551-8d98-dfd6da71fa6c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:24.776149 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:24.786552 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1084/2147] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:20:24 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:20:24.793322 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47926ac0-3a2d-4551-8d98-dfd6da71fa6c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:24.829455 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-06e73fca-dc97-45f4-9414-9db183feba47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '046429fd547f41ae90b5354dfd8b075b', 'request_id': 'req-06e73fca-dc97-45f4-9414-9db183feba47'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:24.843731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-06e73fca-dc97-45f4-9414-9db183feba47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '046429fd547f41ae90b5354dfd8b075b', 'request_id': 'req-06e73fca-dc97-45f4-9414-9db183feba47'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:24.849742 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-06e73fca-dc97-45f4-9414-9db183feba47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '046429fd547f41ae90b5354dfd8b075b', 'request_id': 'req-06e73fca-dc97-45f4-9414-9db183feba47'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:24.866846 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-06e73fca-dc97-45f4-9414-9db183feba47 admin admin] The token cache is being invalidate because project 046429fd547f41ae90b5354dfd8b075b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:20:24.867265 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-06e73fca-dc97-45f4-9414-9db183feba47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:24.868224 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-06e73fca-dc97-45f4-9414-9db183feba47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:24.870591 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1064/2148] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:20:24 2022] DELETE /identity/v3/projects/046429fd547f41ae90b5354dfd8b075b => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:24.874911 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e73fca-dc97-45f4-9414-9db183feba47 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:24.877683 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-06e73fca-dc97-45f4-9414-9db183feba47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:24.889504 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47926ac0-3a2d-4551-8d98-dfd6da71fa6c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:24.894308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:24.959904 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5310798e-4904-4c18-a650-3cb20716670b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:24.960793 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5310798e-4904-4c18-a650-3cb20716670b admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:24.961164 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5310798e-4904-4c18-a650-3cb20716670b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:24.961543 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5310798e-4904-4c18-a650-3cb20716670b admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:24.962656 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5310798e-4904-4c18-a650-3cb20716670b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:24.964856 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5310798e-4904-4c18-a650-3cb20716670b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:24.965831 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e73fca-dc97-45f4-9414-9db183feba47 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:24.971606 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60c2481-01c4-4d04-b8c7-7eb697ebc37b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:24.972242 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c2481-01c4-4d04-b8c7-7eb697ebc37b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:24.972575 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c2481-01c4-4d04-b8c7-7eb697ebc37b admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:24.972921 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c2481-01c4-4d04-b8c7-7eb697ebc37b admin admin] PATH_INFO: `/v3/users/89f2cc637f044241b4b754cc3871e314` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:24.974588 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60c2481-01c4-4d04-b8c7-7eb697ebc37b admin admin] RBAC: Authorizing `identity:delete_user(user_id=89f2cc637f044241b4b754cc3871e314)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:24.978493 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60c2481-01c4-4d04-b8c7-7eb697ebc37b admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:24.990036 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1085/2149] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:20:24 2022] POST /identity/v3/users => generated 298 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:20:24.993807 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5310798e-4904-4c18-a650-3cb20716670b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:24.995646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5310798e-4904-4c18-a650-3cb20716670b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:25.008656 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5310798e-4904-4c18-a650-3cb20716670b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:25.015706 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb19db-e1f0-4521-b96c-6c45f5c10634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:25.016525 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb19db-e1f0-4521-b96c-6c45f5c10634 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:25.016804 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb19db-e1f0-4521-b96c-6c45f5c10634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:25.017543 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb19db-e1f0-4521-b96c-6c45f5c10634 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:25.018303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defb19db-e1f0-4521-b96c-6c45f5c10634 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:25.019989 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defb19db-e1f0-4521-b96c-6c45f5c10634 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:25.031004 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1086/2150] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:24 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:20:25.035154 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb19db-e1f0-4521-b96c-6c45f5c10634 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:25.037238 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a60c2481-01c4-4d04-b8c7-7eb697ebc37b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89f2cc637f044241b4b754cc3871e314'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:25.045928 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a60c2481-01c4-4d04-b8c7-7eb697ebc37b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89f2cc637f044241b4b754cc3871e314'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:25.046798 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a60c2481-01c4-4d04-b8c7-7eb697ebc37b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89f2cc637f044241b4b754cc3871e314'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:25.047967 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb19db-e1f0-4521-b96c-6c45f5c10634 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:25.053756 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c27d8c2-7974-41bf-b6c1-900a67b0e969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:25.054869 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c27d8c2-7974-41bf-b6c1-900a67b0e969 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:25.055189 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c27d8c2-7974-41bf-b6c1-900a67b0e969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:25.055560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c27d8c2-7974-41bf-b6c1-900a67b0e969 admin admin] PATH_INFO: `/v3/projects/89fa07d2cb6b4373b32159a653faf6e2/users/ee86a34437cf417094efbacb77605a83/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:25.057863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a60c2481-01c4-4d04-b8c7-7eb697ebc37b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89f2cc637f044241b4b754cc3871e314'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:25.058822 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a60c2481-01c4-4d04-b8c7-7eb697ebc37b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89f2cc637f044241b4b754cc3871e314'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:25.072419 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c27d8c2-7974-41bf-b6c1-900a67b0e969 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89fa07d2cb6b4373b32159a653faf6e2, user_id=ee86a34437cf417094efbacb77605a83, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:25.075168 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c27d8c2-7974-41bf-b6c1-900a67b0e969 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:25.080600 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1065/2151] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:20:24 2022] DELETE /identity/v3/users/89f2cc637f044241b4b754cc3871e314 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:20:25.089265 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1087/2152] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:20:25 2022] PUT /identity/v3/projects/89fa07d2cb6b4373b32159a653faf6e2/users/ee86a34437cf417094efbacb77605a83/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:20:25.093242 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60c2481-01c4-4d04-b8c7-7eb697ebc37b admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:25.094910 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a60c2481-01c4-4d04-b8c7-7eb697ebc37b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:25.147721 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:25.182831 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60c2481-01c4-4d04-b8c7-7eb697ebc37b admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:25.288768 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f6523d-d48d-4cf8-ac34-cd58c35faaa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:25.290251 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6523d-d48d-4cf8-ac34-cd58c35faaa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:25.290251 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6523d-d48d-4cf8-ac34-cd58c35faaa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:25.290251 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6523d-d48d-4cf8-ac34-cd58c35faaa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:25.290618 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f6523d-d48d-4cf8-ac34-cd58c35faaa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:25.293467 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f6523d-d48d-4cf8-ac34-cd58c35faaa1 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:25.304429 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1066/2153] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:20:25.309365 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c27d8c2-7974-41bf-b6c1-900a67b0e969 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:25.322681 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c27d8c2-7974-41bf-b6c1-900a67b0e969 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:25.328710 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb3c6d-b089-4144-9f9b-2007ae86c861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:25.329420 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb3c6d-b089-4144-9f9b-2007ae86c861 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:25.330306 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb3c6d-b089-4144-9f9b-2007ae86c861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:25.330595 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb3c6d-b089-4144-9f9b-2007ae86c861 admin admin] PATH_INFO: `/v3/projects/89fa07d2cb6b4373b32159a653faf6e2/users/ee86a34437cf417094efbacb77605a83/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:25.333619 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bb3c6d-b089-4144-9f9b-2007ae86c861 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89fa07d2cb6b4373b32159a653faf6e2, user_id=ee86a34437cf417094efbacb77605a83, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:25.337868 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bb3c6d-b089-4144-9f9b-2007ae86c861 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:25.358241 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1088/2154] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:20:25 2022] PUT /identity/v3/projects/89fa07d2cb6b4373b32159a653faf6e2/users/ee86a34437cf417094efbacb77605a83/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:25.364701 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c4b57-35ac-41d7-86bf-4653d15f9191 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:25.364990 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c4b57-35ac-41d7-86bf-4653d15f9191 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:25.365258 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c4b57-35ac-41d7-86bf-4653d15f9191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:25.399132 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-a61c4b57-35ac-41d7-86bf-4653d15f9191 None None] MFA Rules not processed for user `ee86a34437cf417094efbacb77605a83`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:25.400433 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a61c4b57-35ac-41d7-86bf-4653d15f9191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:25.499898 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1067/2155] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:25 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:25.526624 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb3c6d-b089-4144-9f9b-2007ae86c861 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:25.529918 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c5bb3c6d-b089-4144-9f9b-2007ae86c861 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:25.535980 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c4b57-35ac-41d7-86bf-4653d15f9191 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:25.538689 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a61c4b57-35ac-41d7-86bf-4653d15f9191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:25.616520 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb3c6d-b089-4144-9f9b-2007ae86c861 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:25.622298 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4147cc-c7ec-493f-9732-5abb81dedf7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:25.623470 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4147cc-c7ec-493f-9732-5abb81dedf7b service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:25.623681 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4147cc-c7ec-493f-9732-5abb81dedf7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:25.623985 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4147cc-c7ec-493f-9732-5abb81dedf7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:25.638794 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4147cc-c7ec-493f-9732-5abb81dedf7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:25.642043 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4147cc-c7ec-493f-9732-5abb81dedf7b service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:25.655405 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1089/2156] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:20:25 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:20:25.660380 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c4b57-35ac-41d7-86bf-4653d15f9191 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:25.666357 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bdb258-75f9-41dc-a675-f938e636a6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:25.667009 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bdb258-75f9-41dc-a675-f938e636a6df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:25.667383 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bdb258-75f9-41dc-a675-f938e636a6df admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:25.667588 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bdb258-75f9-41dc-a675-f938e636a6df admin admin] PATH_INFO: `/v3/projects/f8505df843904d4aa97771a0a4b4cb5f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:25.668796 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bdb258-75f9-41dc-a675-f938e636a6df admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8505df843904d4aa97771a0a4b4cb5f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:25.670992 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bdb258-75f9-41dc-a675-f938e636a6df admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:25.673282 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:25.755014 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-02bdb258-75f9-41dc-a675-f938e636a6df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8505df843904d4aa97771a0a4b4cb5f', 'request_id': 'req-02bdb258-75f9-41dc-a675-f938e636a6df'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:25.763840 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-02bdb258-75f9-41dc-a675-f938e636a6df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8505df843904d4aa97771a0a4b4cb5f', 'request_id': 'req-02bdb258-75f9-41dc-a675-f938e636a6df'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:25.766831 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca3b880-ae8d-4897-8fbe-55d689b5cea5 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:25.767116 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca3b880-ae8d-4897-8fbe-55d689b5cea5 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:25.767387 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca3b880-ae8d-4897-8fbe-55d689b5cea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:25.771980 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-02bdb258-75f9-41dc-a675-f938e636a6df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8505df843904d4aa97771a0a4b4cb5f', 'request_id': 'req-02bdb258-75f9-41dc-a675-f938e636a6df'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:25.792672 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-02bdb258-75f9-41dc-a675-f938e636a6df admin admin] The token cache is being invalidate because project f8505df843904d4aa97771a0a4b4cb5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:20:25.792980 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-02bdb258-75f9-41dc-a675-f938e636a6df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:25.794851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-02bdb258-75f9-41dc-a675-f938e636a6df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:25.797013 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1068/2157] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:20:25 2022] DELETE /identity/v3/projects/f8505df843904d4aa97771a0a4b4cb5f => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:20:25.798063 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bdb258-75f9-41dc-a675-f938e636a6df admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:25.800438 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-02bdb258-75f9-41dc-a675-f938e636a6df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:25.806125 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-2ca3b880-ae8d-4897-8fbe-55d689b5cea5 None None] MFA Rules not processed for user `9d7f48e1763e4ff3b77ddb90e65045b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:25.807542 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2ca3b880-ae8d-4897-8fbe-55d689b5cea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:25.892360 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bdb258-75f9-41dc-a675-f938e636a6df admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:25.893783 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:25.897881 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c79cc9e-9be4-4b71-889a-e9b7dff98276 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:25.899794 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79cc9e-9be4-4b71-889a-e9b7dff98276 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:25.900242 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79cc9e-9be4-4b71-889a-e9b7dff98276 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:25.900614 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79cc9e-9be4-4b71-889a-e9b7dff98276 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:25.904997 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-7c79cc9e-9be4-4b71-889a-e9b7dff98276 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:25.931690 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1090/2158] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:20:25.933446 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9807e839-56e3-4716-ba2d-a484661c6d0d None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:25.933836 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9807e839-56e3-4716-ba2d-a484661c6d0d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:25.934296 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9807e839-56e3-4716-ba2d-a484661c6d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:25.961360 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-9807e839-56e3-4716-ba2d-a484661c6d0d None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:25.963092 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9807e839-56e3-4716-ba2d-a484661c6d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:25.987745 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c79cc9e-9be4-4b71-889a-e9b7dff98276 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:25.991791 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c79cc9e-9be4-4b71-889a-e9b7dff98276 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:26.007842 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1069/2159] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:20:25 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:20:26.016876 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c79cc9e-9be4-4b71-889a-e9b7dff98276 service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:26.018691 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-7c79cc9e-9be4-4b71-889a-e9b7dff98276 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:26.030880 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c79cc9e-9be4-4b71-889a-e9b7dff98276 service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:26.037502 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726124e8-6e5e-43d9-bd5a-93d0b4f7fe57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:26.038252 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726124e8-6e5e-43d9-bd5a-93d0b4f7fe57 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:26.039100 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726124e8-6e5e-43d9-bd5a-93d0b4f7fe57 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:26.039384 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726124e8-6e5e-43d9-bd5a-93d0b4f7fe57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:26.051108 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726124e8-6e5e-43d9-bd5a-93d0b4f7fe57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:26.052552 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726124e8-6e5e-43d9-bd5a-93d0b4f7fe57 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:26.056514 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1091/2160] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:26.063258 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1070/2161] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:20:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 05:20:26.066270 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db02bfd-ddcf-4f69-90eb-7063efbea0d9 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:26.067116 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db02bfd-ddcf-4f69-90eb-7063efbea0d9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:26.067116 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db02bfd-ddcf-4f69-90eb-7063efbea0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:26.090159 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-9db02bfd-ddcf-4f69-90eb-7063efbea0d9 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:26.091534 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9db02bfd-ddcf-4f69-90eb-7063efbea0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:26.114250 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1092/2162] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:26.119598 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726124e8-6e5e-43d9-bd5a-93d0b4f7fe57 service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:26.141728 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726124e8-6e5e-43d9-bd5a-93d0b4f7fe57 service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:26.151872 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527b3e7-8c03-43be-9edd-c0b339e9596b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:26.152842 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527b3e7-8c03-43be-9edd-c0b339e9596b admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:26.154410 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527b3e7-8c03-43be-9edd-c0b339e9596b admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:26.155088 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527b3e7-8c03-43be-9edd-c0b339e9596b admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:26.156115 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527b3e7-8c03-43be-9edd-c0b339e9596b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:20:26.156835 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527b3e7-8c03-43be-9edd-c0b339e9596b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:26.158859 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527b3e7-8c03-43be-9edd-c0b339e9596b admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:26.178675 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db02bfd-ddcf-4f69-90eb-7063efbea0d9 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:26.185745 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1071/2163] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:20:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:20:26.198455 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db02bfd-ddcf-4f69-90eb-7063efbea0d9 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:26.209731 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88dab6d-7f53-44b0-a819-dbe05ff81663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:26.211720 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88dab6d-7f53-44b0-a819-dbe05ff81663 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:26.212767 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88dab6d-7f53-44b0-a819-dbe05ff81663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:26.213129 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88dab6d-7f53-44b0-a819-dbe05ff81663 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:26.214969 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88dab6d-7f53-44b0-a819-dbe05ff81663 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:26.217913 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88dab6d-7f53-44b0-a819-dbe05ff81663 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:26.268341 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1093/2164] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:20:26 2022] POST /identity/v3/projects => generated 379 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:20:26.270118 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527b3e7-8c03-43be-9edd-c0b339e9596b admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:26.283268 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527b3e7-8c03-43be-9edd-c0b339e9596b admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:26.306826 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:26.379915 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66789a-8518-4d05-8967-4f2e80bb5994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:26.380582 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66789a-8518-4d05-8967-4f2e80bb5994 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:26.380854 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66789a-8518-4d05-8967-4f2e80bb5994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:26.381120 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66789a-8518-4d05-8967-4f2e80bb5994 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:26.381889 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b66789a-8518-4d05-8967-4f2e80bb5994 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:26.383941 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b66789a-8518-4d05-8967-4f2e80bb5994 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:26.411766 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1072/2165] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:20:26 2022] POST /identity/v3/users => generated 290 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:20:26.415872 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88dab6d-7f53-44b0-a819-dbe05ff81663 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:26.430704 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88dab6d-7f53-44b0-a819-dbe05ff81663 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:26.435977 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df57fc1-b6ba-40cd-9c7a-5be64929d6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:26.436700 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df57fc1-b6ba-40cd-9c7a-5be64929d6c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:26.437125 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df57fc1-b6ba-40cd-9c7a-5be64929d6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:26.438248 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df57fc1-b6ba-40cd-9c7a-5be64929d6c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:26.439279 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df57fc1-b6ba-40cd-9c7a-5be64929d6c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:26.440673 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df57fc1-b6ba-40cd-9c7a-5be64929d6c6 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:26.452015 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1094/2166] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:20:26.456567 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66789a-8518-4d05-8967-4f2e80bb5994 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:26.479978 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66789a-8518-4d05-8967-4f2e80bb5994 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:26.485837 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef66a3-e617-4df3-8392-673193e838e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:26.486562 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef66a3-e617-4df3-8392-673193e838e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:26.487209 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef66a3-e617-4df3-8392-673193e838e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:26.487585 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef66a3-e617-4df3-8392-673193e838e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:26.489297 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef66a3-e617-4df3-8392-673193e838e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:26.491492 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef66a3-e617-4df3-8392-673193e838e4 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:26.512210 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1073/2167] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:20:26.516371 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df57fc1-b6ba-40cd-9c7a-5be64929d6c6 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:26.530319 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df57fc1-b6ba-40cd-9c7a-5be64929d6c6 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:26.546263 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412aa4a5-c69f-483e-bdfe-8a0008c0e42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:26.546263 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412aa4a5-c69f-483e-bdfe-8a0008c0e42e admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:26.546263 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412aa4a5-c69f-483e-bdfe-8a0008c0e42e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:26.546263 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412aa4a5-c69f-483e-bdfe-8a0008c0e42e admin admin] PATH_INFO: `/v3/projects/ce4e297be0b2485ebc2673b029067b5f/users/ad258275cb964f34bf59f9360724ce95/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:26.565124 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412aa4a5-c69f-483e-bdfe-8a0008c0e42e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce4e297be0b2485ebc2673b029067b5f, user_id=ad258275cb964f34bf59f9360724ce95, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:26.569247 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412aa4a5-c69f-483e-bdfe-8a0008c0e42e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:26.603190 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1095/2168] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:20:26 2022] PUT /identity/v3/projects/ce4e297be0b2485ebc2673b029067b5f/users/ad258275cb964f34bf59f9360724ce95/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:26.613960 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5fbfb-3d81-452b-8945-99550b3777bc None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:26.614286 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5fbfb-3d81-452b-8945-99550b3777bc None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:26.614564 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5fbfb-3d81-452b-8945-99550b3777bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:26.648818 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-21f5fbfb-3d81-452b-8945-99550b3777bc None None] MFA Rules not processed for user `ad258275cb964f34bf59f9360724ce95`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:26.650069 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-21f5fbfb-3d81-452b-8945-99550b3777bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:26.717366 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1074/2169] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:26.721911 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412aa4a5-c69f-483e-bdfe-8a0008c0e42e admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:26.787558 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412aa4a5-c69f-483e-bdfe-8a0008c0e42e admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:26.793230 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048c10ce-1a84-4cb1-9958-bafe9db73f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:26.793922 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c10ce-1a84-4cb1-9958-bafe9db73f0d admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:26.794248 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c10ce-1a84-4cb1-9958-bafe9db73f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:26.794516 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c10ce-1a84-4cb1-9958-bafe9db73f0d admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:26.795159 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048c10ce-1a84-4cb1-9958-bafe9db73f0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:26.797061 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048c10ce-1a84-4cb1-9958-bafe9db73f0d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:26.828189 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1096/2170] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:20:26 2022] POST /identity/v3/projects => generated 379 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:20:26.829442 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f5fbfb-3d81-452b-8945-99550b3777bc None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:26.841840 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f5fbfb-3d81-452b-8945-99550b3777bc None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:26.903625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeafba5-b775-4774-89fb-1cef1cecc0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:26.904267 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeafba5-b775-4774-89fb-1cef1cecc0aa admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:26.904564 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeafba5-b775-4774-89fb-1cef1cecc0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:26.904836 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeafba5-b775-4774-89fb-1cef1cecc0aa admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:26.905495 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeafba5-b775-4774-89fb-1cef1cecc0aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:26.908077 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeafba5-b775-4774-89fb-1cef1cecc0aa admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:26.933956 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1075/2171] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:20:26 2022] POST /identity/v3/users => generated 290 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:20:26.942102 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048c10ce-1a84-4cb1-9958-bafe9db73f0d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:26.943999 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:26.952706 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048c10ce-1a84-4cb1-9958-bafe9db73f0d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:26.957893 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414f6f5-cc36-49d3-96a9-050c7799663b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:26.958527 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414f6f5-cc36-49d3-96a9-050c7799663b admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:26.958800 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414f6f5-cc36-49d3-96a9-050c7799663b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:26.959065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414f6f5-cc36-49d3-96a9-050c7799663b admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:26.959796 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c414f6f5-cc36-49d3-96a9-050c7799663b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:26.961400 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c414f6f5-cc36-49d3-96a9-050c7799663b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:26.972674 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1097/2172] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:20:26.977331 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeafba5-b775-4774-89fb-1cef1cecc0aa admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:26.990119 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeafba5-b775-4774-89fb-1cef1cecc0aa admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:26.995533 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d6b568-892c-479d-8fb8-4f2c01a5b9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:26.996098 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d6b568-892c-479d-8fb8-4f2c01a5b9e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:26.996471 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d6b568-892c-479d-8fb8-4f2c01a5b9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:26.996759 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d6b568-892c-479d-8fb8-4f2c01a5b9e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:26.997455 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d6b568-892c-479d-8fb8-4f2c01a5b9e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:26.999628 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d6b568-892c-479d-8fb8-4f2c01a5b9e1 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:27.012871 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1076/2173] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:20:27.016793 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414f6f5-cc36-49d3-96a9-050c7799663b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:27.027211 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414f6f5-cc36-49d3-96a9-050c7799663b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:27.031926 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b3c2f-d962-4273-99cd-97cc34198fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:27.032580 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b3c2f-d962-4273-99cd-97cc34198fe8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:27.032856 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b3c2f-d962-4273-99cd-97cc34198fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:27.033140 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b3c2f-d962-4273-99cd-97cc34198fe8 admin admin] PATH_INFO: `/v3/projects/a031059f951b4f1aa003a1ee51833e4d/users/fae438d1761643f3b7f6d1023ce5f6a7/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:27.055515 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451b3c2f-d962-4273-99cd-97cc34198fe8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a031059f951b4f1aa003a1ee51833e4d, user_id=fae438d1761643f3b7f6d1023ce5f6a7, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:27.058491 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451b3c2f-d962-4273-99cd-97cc34198fe8 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:27.073450 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1098/2174] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:20:27 2022] PUT /identity/v3/projects/a031059f951b4f1aa003a1ee51833e4d/users/fae438d1761643f3b7f6d1023ce5f6a7/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:20:27.085090 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745b4fd-9813-4543-a482-ed62d42b8861 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:27.085569 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745b4fd-9813-4543-a482-ed62d42b8861 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:27.085677 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745b4fd-9813-4543-a482-ed62d42b8861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:27.094685 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:27.117424 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-9745b4fd-9813-4543-a482-ed62d42b8861 None None] MFA Rules not processed for user `fae438d1761643f3b7f6d1023ce5f6a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:27.118925 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-9745b4fd-9813-4543-a482-ed62d42b8861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:27.184091 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1077/2175] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:27.189295 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b3c2f-d962-4273-99cd-97cc34198fe8 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:27.263911 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b3c2f-d962-4273-99cd-97cc34198fe8 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:27.272862 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217222b2-ec1b-4dee-ac72-1d129f5f92ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:27.273657 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217222b2-ec1b-4dee-ac72-1d129f5f92ac admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:27.273951 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217222b2-ec1b-4dee-ac72-1d129f5f92ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:27.274327 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217222b2-ec1b-4dee-ac72-1d129f5f92ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:27.275258 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217222b2-ec1b-4dee-ac72-1d129f5f92ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:27.278170 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217222b2-ec1b-4dee-ac72-1d129f5f92ac admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:27.314613 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1099/2176] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:20:27 2022] POST /identity/v3/projects => generated 379 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:20:27.320986 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745b4fd-9813-4543-a482-ed62d42b8861 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:27.335797 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745b4fd-9813-4543-a482-ed62d42b8861 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:27.396467 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ec2c45-2240-404d-bb8c-3c45090d033d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:27.397072 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec2c45-2240-404d-bb8c-3c45090d033d admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:27.397346 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec2c45-2240-404d-bb8c-3c45090d033d admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:27.397609 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec2c45-2240-404d-bb8c-3c45090d033d admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:27.398266 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ec2c45-2240-404d-bb8c-3c45090d033d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:27.400193 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ec2c45-2240-404d-bb8c-3c45090d033d admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:27.422971 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1078/2177] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:20:27 2022] POST /identity/v3/users => generated 296 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:20:27.427166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217222b2-ec1b-4dee-ac72-1d129f5f92ac admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:27.438159 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217222b2-ec1b-4dee-ac72-1d129f5f92ac admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:27.443832 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5728f2ff-069a-4c4f-9ac5-1b0e70a70d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:27.444430 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5728f2ff-069a-4c4f-9ac5-1b0e70a70d7c admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:27.444707 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5728f2ff-069a-4c4f-9ac5-1b0e70a70d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:27.444971 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5728f2ff-069a-4c4f-9ac5-1b0e70a70d7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:27.445616 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5728f2ff-069a-4c4f-9ac5-1b0e70a70d7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:27.447258 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5728f2ff-069a-4c4f-9ac5-1b0e70a70d7c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:27.458439 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1100/2178] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:20:27.462808 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ec2c45-2240-404d-bb8c-3c45090d033d admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:27.473594 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ec2c45-2240-404d-bb8c-3c45090d033d admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:27.479934 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a964d-5a31-4de9-b163-dd1470625ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:27.480687 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a964d-5a31-4de9-b163-dd1470625ad4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:27.480970 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a964d-5a31-4de9-b163-dd1470625ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:27.481268 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a964d-5a31-4de9-b163-dd1470625ad4 admin admin] PATH_INFO: `/v3/projects/dfbd62d6cb104e7b9e1c9ed5dcacdebd/users/cd229338453346fdad8ee6673b49616e/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:27.498612 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44a964d-5a31-4de9-b163-dd1470625ad4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfbd62d6cb104e7b9e1c9ed5dcacdebd, user_id=cd229338453346fdad8ee6673b49616e, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:27.501432 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44a964d-5a31-4de9-b163-dd1470625ad4 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:27.516550 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1079/2179] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:20:27 2022] PUT /identity/v3/projects/dfbd62d6cb104e7b9e1c9ed5dcacdebd/users/cd229338453346fdad8ee6673b49616e/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:27.521018 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5728f2ff-069a-4c4f-9ac5-1b0e70a70d7c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:27.531883 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5728f2ff-069a-4c4f-9ac5-1b0e70a70d7c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:27.597086 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf656e98-248d-47e9-a39c-f6db5f658540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:27.597744 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf656e98-248d-47e9-a39c-f6db5f658540 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:27.598033 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf656e98-248d-47e9-a39c-f6db5f658540 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:27.598628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf656e98-248d-47e9-a39c-f6db5f658540 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:27.599272 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf656e98-248d-47e9-a39c-f6db5f658540 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:27.602293 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf656e98-248d-47e9-a39c-f6db5f658540 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:27.617618 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1101/2180] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:27 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:20:27.622887 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a964d-5a31-4de9-b163-dd1470625ad4 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:27.634081 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a964d-5a31-4de9-b163-dd1470625ad4 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:27.644400 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fef2ea2-6ccc-416a-8071-ca4a08be2faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:27.645557 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef2ea2-6ccc-416a-8071-ca4a08be2faa admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:27.645557 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef2ea2-6ccc-416a-8071-ca4a08be2faa admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:27.645770 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef2ea2-6ccc-416a-8071-ca4a08be2faa admin admin] PATH_INFO: `/v3/projects/dfbd62d6cb104e7b9e1c9ed5dcacdebd/users/cd229338453346fdad8ee6673b49616e/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:27.648218 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fef2ea2-6ccc-416a-8071-ca4a08be2faa admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfbd62d6cb104e7b9e1c9ed5dcacdebd, user_id=cd229338453346fdad8ee6673b49616e, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:27.651194 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fef2ea2-6ccc-416a-8071-ca4a08be2faa admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:27.668342 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1080/2181] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:20:27 2022] PUT /identity/v3/projects/dfbd62d6cb104e7b9e1c9ed5dcacdebd/users/cd229338453346fdad8ee6673b49616e/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:27.674280 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36653d-80b2-42e7-9b65-72fdc7e9e803 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:27.674562 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36653d-80b2-42e7-9b65-72fdc7e9e803 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:27.674828 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36653d-80b2-42e7-9b65-72fdc7e9e803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:27.707957 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-bd36653d-80b2-42e7-9b65-72fdc7e9e803 None None] MFA Rules not processed for user `cd229338453346fdad8ee6673b49616e`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:27.709297 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-bd36653d-80b2-42e7-9b65-72fdc7e9e803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:27.745169 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:27.815399 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1102/2182] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:27 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:27.861043 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fef2ea2-6ccc-416a-8071-ca4a08be2faa admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:27.921822 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:27.928723 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fef2ea2-6ccc-416a-8071-ca4a08be2faa admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:27.932598 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6773c6-6016-44fb-b776-f293db649c23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:27.933675 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6773c6-6016-44fb-b776-f293db649c23 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:27.933974 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6773c6-6016-44fb-b776-f293db649c23 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:27.934432 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6773c6-6016-44fb-b776-f293db649c23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:27.948004 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6773c6-6016-44fb-b776-f293db649c23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:27.949664 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6773c6-6016-44fb-b776-f293db649c23 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:27.960648 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1081/2183] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:20:27 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:20:28.005368 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd36653d-80b2-42e7-9b65-72fdc7e9e803 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:28.015691 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd36653d-80b2-42e7-9b65-72fdc7e9e803 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:28.020135 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c51b54-b18b-4161-aa81-61edf385dcef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:28.020737 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c51b54-b18b-4161-aa81-61edf385dcef service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:28.021013 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c51b54-b18b-4161-aa81-61edf385dcef service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:28.021277 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c51b54-b18b-4161-aa81-61edf385dcef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:28.089138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c51b54-b18b-4161-aa81-61edf385dcef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:28.090667 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c51b54-b18b-4161-aa81-61edf385dcef service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:28.100196 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1103/2184] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:20:28 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:20:28.673503 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c672e-e55f-4db5-bf38-31f84e12cc95 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:28.674056 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c672e-e55f-4db5-bf38-31f84e12cc95 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:28.674056 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c672e-e55f-4db5-bf38-31f84e12cc95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:28.694064 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-9d9c672e-e55f-4db5-bf38-31f84e12cc95 None None] MFA Rules not processed for user `9d7f48e1763e4ff3b77ddb90e65045b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:28.695265 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-9d9c672e-e55f-4db5-bf38-31f84e12cc95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:28.751991 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1082/2185] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:28.946804 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c51b54-b18b-4161-aa81-61edf385dcef service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:28.948688 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b6c51b54-b18b-4161-aa81-61edf385dcef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:29.026918 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c51b54-b18b-4161-aa81-61edf385dcef service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:29.031530 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c28b00a-067c-46db-9914-16e4952b93be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:29.032252 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28b00a-067c-46db-9914-16e4952b93be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:29.032530 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28b00a-067c-46db-9914-16e4952b93be admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:29.032800 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28b00a-067c-46db-9914-16e4952b93be admin admin] PATH_INFO: `/v3/users/9dd8e02e25c1461f8aee21589271d4b8` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:29.033934 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c28b00a-067c-46db-9914-16e4952b93be admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dd8e02e25c1461f8aee21589271d4b8)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:29.037043 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c28b00a-067c-46db-9914-16e4952b93be admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:29.102076 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5c28b00a-067c-46db-9914-16e4952b93be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dd8e02e25c1461f8aee21589271d4b8'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:29.110613 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5c28b00a-067c-46db-9914-16e4952b93be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dd8e02e25c1461f8aee21589271d4b8'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:29.119246 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5c28b00a-067c-46db-9914-16e4952b93be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dd8e02e25c1461f8aee21589271d4b8'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:29.119855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5c28b00a-067c-46db-9914-16e4952b93be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dd8e02e25c1461f8aee21589271d4b8'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:29.121069 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:29.134781 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5c28b00a-067c-46db-9914-16e4952b93be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dd8e02e25c1461f8aee21589271d4b8'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:29.141608 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1104/2186] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:20:28 2022] DELETE /identity/v3/users/9dd8e02e25c1461f8aee21589271d4b8 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:29.146302 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8320c83-0631-4cf9-bbf5-be23e1a3ba6d None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:29.146772 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8320c83-0631-4cf9-bbf5-be23e1a3ba6d None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:29.147017 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8320c83-0631-4cf9-bbf5-be23e1a3ba6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:29.166733 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-a8320c83-0631-4cf9-bbf5-be23e1a3ba6d None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:29.168003 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a8320c83-0631-4cf9-bbf5-be23e1a3ba6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:29.258262 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1083/2187] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:29.268528 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c28b00a-067c-46db-9914-16e4952b93be admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:29.270380 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5c28b00a-067c-46db-9914-16e4952b93be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:29.333105 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c28b00a-067c-46db-9914-16e4952b93be admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:29.339512 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1ca8da-1bb8-4abc-a0fe-708b1a7876e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:29.340239 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ca8da-1bb8-4abc-a0fe-708b1a7876e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:29.340518 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ca8da-1bb8-4abc-a0fe-708b1a7876e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:29.340786 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ca8da-1bb8-4abc-a0fe-708b1a7876e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:29.350466 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1ca8da-1bb8-4abc-a0fe-708b1a7876e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:29.351911 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1ca8da-1bb8-4abc-a0fe-708b1a7876e3 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:29.361859 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1105/2188] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:20:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:20:30.166693 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8320c83-0631-4cf9-bbf5-be23e1a3ba6d None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:30.178163 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a8320c83-0631-4cf9-bbf5-be23e1a3ba6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:30.212127 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8320c83-0631-4cf9-bbf5-be23e1a3ba6d None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:30.229036 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845475a8-78e6-4e3f-b052-c459107c266e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:30.229704 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845475a8-78e6-4e3f-b052-c459107c266e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:30.229959 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845475a8-78e6-4e3f-b052-c459107c266e admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:30.230251 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845475a8-78e6-4e3f-b052-c459107c266e admin admin] PATH_INFO: `/v3/projects/c1a8a60fc87643d4ae48b72a45191de3` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:30.235688 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845475a8-78e6-4e3f-b052-c459107c266e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1a8a60fc87643d4ae48b72a45191de3)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:30.237672 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845475a8-78e6-4e3f-b052-c459107c266e admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:30.290979 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-845475a8-78e6-4e3f-b052-c459107c266e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1a8a60fc87643d4ae48b72a45191de3', 'request_id': 'req-845475a8-78e6-4e3f-b052-c459107c266e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:30.297867 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-845475a8-78e6-4e3f-b052-c459107c266e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1a8a60fc87643d4ae48b72a45191de3', 'request_id': 'req-845475a8-78e6-4e3f-b052-c459107c266e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:30.303556 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-845475a8-78e6-4e3f-b052-c459107c266e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1a8a60fc87643d4ae48b72a45191de3', 'request_id': 'req-845475a8-78e6-4e3f-b052-c459107c266e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:30.316229 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-845475a8-78e6-4e3f-b052-c459107c266e admin admin] The token cache is being invalidate because project c1a8a60fc87643d4ae48b72a45191de3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:20:30.316509 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-845475a8-78e6-4e3f-b052-c459107c266e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:30.317117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-845475a8-78e6-4e3f-b052-c459107c266e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:30.318952 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1084/2189] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:20:30 2022] DELETE /identity/v3/projects/c1a8a60fc87643d4ae48b72a45191de3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:30.322892 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1ca8da-1bb8-4abc-a0fe-708b1a7876e3 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:30.332542 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1ca8da-1bb8-4abc-a0fe-708b1a7876e3 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:30.389462 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee77f04f-a9b1-4224-bbcc-4fcedd54bcba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:30.390019 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee77f04f-a9b1-4224-bbcc-4fcedd54bcba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:30.390306 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee77f04f-a9b1-4224-bbcc-4fcedd54bcba admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:30.390590 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee77f04f-a9b1-4224-bbcc-4fcedd54bcba admin admin] PATH_INFO: `/v3/users/ee86a34437cf417094efbacb77605a83` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:30.391702 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee77f04f-a9b1-4224-bbcc-4fcedd54bcba admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee86a34437cf417094efbacb77605a83)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:30.393627 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee77f04f-a9b1-4224-bbcc-4fcedd54bcba admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:30.443155 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ee77f04f-a9b1-4224-bbcc-4fcedd54bcba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee86a34437cf417094efbacb77605a83'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:30.450136 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ee77f04f-a9b1-4224-bbcc-4fcedd54bcba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee86a34437cf417094efbacb77605a83'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:30.459665 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ee77f04f-a9b1-4224-bbcc-4fcedd54bcba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee86a34437cf417094efbacb77605a83'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:30.460306 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ee77f04f-a9b1-4224-bbcc-4fcedd54bcba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee86a34437cf417094efbacb77605a83'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:30.472807 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ee77f04f-a9b1-4224-bbcc-4fcedd54bcba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee86a34437cf417094efbacb77605a83'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:30.478647 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1106/2190] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:20:30 2022] DELETE /identity/v3/users/ee86a34437cf417094efbacb77605a83 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:30.968688 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845475a8-78e6-4e3f-b052-c459107c266e admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:30.970725 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-845475a8-78e6-4e3f-b052-c459107c266e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:30.998130 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845475a8-78e6-4e3f-b052-c459107c266e admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:31.020581 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:31.098276 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf347e6-a9da-4e74-985d-06918f4d0a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:31.098923 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf347e6-a9da-4e74-985d-06918f4d0a6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:31.099199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf347e6-a9da-4e74-985d-06918f4d0a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:31.100338 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf347e6-a9da-4e74-985d-06918f4d0a6f admin admin] PATH_INFO: `/v3/projects/89fa07d2cb6b4373b32159a653faf6e2` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:31.101803 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf347e6-a9da-4e74-985d-06918f4d0a6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=89fa07d2cb6b4373b32159a653faf6e2)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:31.103966 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf347e6-a9da-4e74-985d-06918f4d0a6f admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:31.169322 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-abf347e6-a9da-4e74-985d-06918f4d0a6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89fa07d2cb6b4373b32159a653faf6e2', 'request_id': 'req-abf347e6-a9da-4e74-985d-06918f4d0a6f'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:31.178748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-abf347e6-a9da-4e74-985d-06918f4d0a6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89fa07d2cb6b4373b32159a653faf6e2', 'request_id': 'req-abf347e6-a9da-4e74-985d-06918f4d0a6f'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:31.188405 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-abf347e6-a9da-4e74-985d-06918f4d0a6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89fa07d2cb6b4373b32159a653faf6e2', 'request_id': 'req-abf347e6-a9da-4e74-985d-06918f4d0a6f'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:31.208757 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-abf347e6-a9da-4e74-985d-06918f4d0a6f admin admin] The token cache is being invalidate because project 89fa07d2cb6b4373b32159a653faf6e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:20:31.209592 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-abf347e6-a9da-4e74-985d-06918f4d0a6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:31.214472 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-abf347e6-a9da-4e74-985d-06918f4d0a6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:31.218136 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1085/2191] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:20:30 2022] DELETE /identity/v3/projects/89fa07d2cb6b4373b32159a653faf6e2 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:31.225255 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61a826-0c4e-480f-a951-66d763bcb32f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:31.226138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61a826-0c4e-480f-a951-66d763bcb32f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:31.227263 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61a826-0c4e-480f-a951-66d763bcb32f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:31.243324 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:31.259268 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-4b61a826-0c4e-480f-a951-66d763bcb32f None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:31.260480 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4b61a826-0c4e-480f-a951-66d763bcb32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:31.333971 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1107/2192] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:31.342782 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5a253-6aa7-4248-bd03-0ac5de09af8e None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:31.343070 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5a253-6aa7-4248-bd03-0ac5de09af8e None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:31.343373 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5a253-6aa7-4248-bd03-0ac5de09af8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:31.365238 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-5bb5a253-6aa7-4248-bd03-0ac5de09af8e None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:31.366505 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-5bb5a253-6aa7-4248-bd03-0ac5de09af8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:31.374592 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1086/2193] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:31.378823 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b61a826-0c4e-480f-a951-66d763bcb32f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:31.388417 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b61a826-0c4e-480f-a951-66d763bcb32f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:31.392886 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352b512-1855-4cf7-a626-28736ec1bca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:31.393481 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352b512-1855-4cf7-a626-28736ec1bca2 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:31.393752 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352b512-1855-4cf7-a626-28736ec1bca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:31.394027 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352b512-1855-4cf7-a626-28736ec1bca2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:31.394689 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1352b512-1855-4cf7-a626-28736ec1bca2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:20:31.394995 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1352b512-1855-4cf7-a626-28736ec1bca2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:31.396583 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1352b512-1855-4cf7-a626-28736ec1bca2 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:31.408707 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1108/2194] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:20:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:20:31.414306 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb5a253-6aa7-4248-bd03-0ac5de09af8e None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:31.426146 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb5a253-6aa7-4248-bd03-0ac5de09af8e None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:31.430762 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a7777-c93c-48b4-a08d-247e3748a0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:31.431309 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a7777-c93c-48b4-a08d-247e3748a0ca admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:31.431589 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a7777-c93c-48b4-a08d-247e3748a0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:31.431986 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a7777-c93c-48b4-a08d-247e3748a0ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:31.432646 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999a7777-c93c-48b4-a08d-247e3748a0ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:31.434663 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999a7777-c93c-48b4-a08d-247e3748a0ca admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:31.463558 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1087/2195] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:20:31 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:20:31.467891 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352b512-1855-4cf7-a626-28736ec1bca2 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:31.479859 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352b512-1855-4cf7-a626-28736ec1bca2 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:31.548409 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631a144-4476-4000-a41a-1ef870dc895d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:31.549038 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631a144-4476-4000-a41a-1ef870dc895d admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:31.549315 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631a144-4476-4000-a41a-1ef870dc895d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:31.549580 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631a144-4476-4000-a41a-1ef870dc895d admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:31.550255 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3631a144-4476-4000-a41a-1ef870dc895d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:31.552334 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3631a144-4476-4000-a41a-1ef870dc895d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:31.575880 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1109/2196] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:20:31 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:20:31.579723 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a7777-c93c-48b4-a08d-247e3748a0ca admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:31.591140 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a7777-c93c-48b4-a08d-247e3748a0ca admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:31.596425 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd4983-70ee-4626-b14e-a98eeeab5b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:31.597165 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd4983-70ee-4626-b14e-a98eeeab5b9c admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:31.597524 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd4983-70ee-4626-b14e-a98eeeab5b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:31.597844 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd4983-70ee-4626-b14e-a98eeeab5b9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:31.598734 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fd4983-70ee-4626-b14e-a98eeeab5b9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:31.600619 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fd4983-70ee-4626-b14e-a98eeeab5b9c admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:31.613613 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1088/2197] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:20:31.617763 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631a144-4476-4000-a41a-1ef870dc895d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:31.628553 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631a144-4476-4000-a41a-1ef870dc895d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:31.633916 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d981414e-ca8a-4e3e-adbb-8edeb51af613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:31.634554 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981414e-ca8a-4e3e-adbb-8edeb51af613 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:31.634835 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981414e-ca8a-4e3e-adbb-8edeb51af613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:31.635102 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981414e-ca8a-4e3e-adbb-8edeb51af613 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:31.635752 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d981414e-ca8a-4e3e-adbb-8edeb51af613 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:31.637423 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d981414e-ca8a-4e3e-adbb-8edeb51af613 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:31.650053 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1110/2198] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:20:31.654194 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd4983-70ee-4626-b14e-a98eeeab5b9c admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:31.673247 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd4983-70ee-4626-b14e-a98eeeab5b9c admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:31.680346 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3931cfa4-1d51-42d7-b086-690d6410fa04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:31.681159 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3931cfa4-1d51-42d7-b086-690d6410fa04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:31.681565 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3931cfa4-1d51-42d7-b086-690d6410fa04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:31.682055 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3931cfa4-1d51-42d7-b086-690d6410fa04 admin admin] PATH_INFO: `/v3/projects/4b93d7cc85a343729d23f81d492e9783/users/f5d99e9b0ac74accb7b2c921920fd4a4/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:31.708447 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3931cfa4-1d51-42d7-b086-690d6410fa04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b93d7cc85a343729d23f81d492e9783, user_id=f5d99e9b0ac74accb7b2c921920fd4a4, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:31.711332 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3931cfa4-1d51-42d7-b086-690d6410fa04 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:31.727309 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1089/2199] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:20:31 2022] PUT /identity/v3/projects/4b93d7cc85a343729d23f81d492e9783/users/f5d99e9b0ac74accb7b2c921920fd4a4/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:31.738160 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a388e08-e0d2-4c80-88e7-bf09c083c245 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:31.738557 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a388e08-e0d2-4c80-88e7-bf09c083c245 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:31.738802 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a388e08-e0d2-4c80-88e7-bf09c083c245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:31.769307 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-4a388e08-e0d2-4c80-88e7-bf09c083c245 None None] MFA Rules not processed for user `f5d99e9b0ac74accb7b2c921920fd4a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:31.770108 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4a388e08-e0d2-4c80-88e7-bf09c083c245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:31.877738 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:31.889428 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1111/2200] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:20:31.900585 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3931cfa4-1d51-42d7-b086-690d6410fa04 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:31.902615 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3931cfa4-1d51-42d7-b086-690d6410fa04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:32.003414 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:32.030285 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3931cfa4-1d51-42d7-b086-690d6410fa04 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:32.036264 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebb4608-a42e-4f43-968a-692e1270f9c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:32.037055 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb4608-a42e-4f43-968a-692e1270f9c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:32.037718 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb4608-a42e-4f43-968a-692e1270f9c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:32.038021 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb4608-a42e-4f43-968a-692e1270f9c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:32.056587 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebb4608-a42e-4f43-968a-692e1270f9c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:32.058905 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebb4608-a42e-4f43-968a-692e1270f9c8 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:32.072386 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1090/2201] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:20:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:20:32.097321 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a388e08-e0d2-4c80-88e7-bf09c083c245 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:32.171572 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a388e08-e0d2-4c80-88e7-bf09c083c245 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:32.181556 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f986026-e921-490e-a853-f609a606f009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:32.182706 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f986026-e921-490e-a853-f609a606f009 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:32.183238 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f986026-e921-490e-a853-f609a606f009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:32.184335 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f986026-e921-490e-a853-f609a606f009 admin admin] PATH_INFO: `/v3/users/f5d99e9b0ac74accb7b2c921920fd4a4` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:32.186772 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f986026-e921-490e-a853-f609a606f009 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5d99e9b0ac74accb7b2c921920fd4a4)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:32.189603 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f986026-e921-490e-a853-f609a606f009 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:32.294184 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1f986026-e921-490e-a853-f609a606f009 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5d99e9b0ac74accb7b2c921920fd4a4'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:32.309267 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1f986026-e921-490e-a853-f609a606f009 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5d99e9b0ac74accb7b2c921920fd4a4'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:32.322664 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1f986026-e921-490e-a853-f609a606f009 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5d99e9b0ac74accb7b2c921920fd4a4'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:32.323623 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1f986026-e921-490e-a853-f609a606f009 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5d99e9b0ac74accb7b2c921920fd4a4'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:32.339538 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1f986026-e921-490e-a853-f609a606f009 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5d99e9b0ac74accb7b2c921920fd4a4'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:32.350717 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1112/2202] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:20:32 2022] DELETE /identity/v3/users/f5d99e9b0ac74accb7b2c921920fd4a4 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:32.354963 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb21df1-2462-4f40-ac3a-fbbf7874d873 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:32.355476 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb21df1-2462-4f40-ac3a-fbbf7874d873 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:32.355575 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb21df1-2462-4f40-ac3a-fbbf7874d873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:32.386185 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-aeb21df1-2462-4f40-ac3a-fbbf7874d873 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:32.388714 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-aeb21df1-2462-4f40-ac3a-fbbf7874d873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:32.440031 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f986026-e921-490e-a853-f609a606f009 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:32.441569 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1f986026-e921-490e-a853-f609a606f009 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:32.527140 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1091/2203] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:32.546453 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb21df1-2462-4f40-ac3a-fbbf7874d873 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:32.548408 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-aeb21df1-2462-4f40-ac3a-fbbf7874d873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:32.585341 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f986026-e921-490e-a853-f609a606f009 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:32.591731 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846bfcd-24a3-4899-b5e0-53f8a7ddf240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:32.591731 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846bfcd-24a3-4899-b5e0-53f8a7ddf240 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:32.591731 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846bfcd-24a3-4899-b5e0-53f8a7ddf240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:32.592071 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846bfcd-24a3-4899-b5e0-53f8a7ddf240 admin admin] PATH_INFO: `/v3/users/ad258275cb964f34bf59f9360724ce95` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:32.593161 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5846bfcd-24a3-4899-b5e0-53f8a7ddf240 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad258275cb964f34bf59f9360724ce95)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:32.595555 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5846bfcd-24a3-4899-b5e0-53f8a7ddf240 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:32.626420 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:32.672148 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb21df1-2462-4f40-ac3a-fbbf7874d873 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:32.674101 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5846bfcd-24a3-4899-b5e0-53f8a7ddf240 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad258275cb964f34bf59f9360724ce95'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:32.676897 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a27790-5bca-4647-9122-15c642b45835 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:32.677892 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a27790-5bca-4647-9122-15c642b45835 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:32.677892 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a27790-5bca-4647-9122-15c642b45835 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:32.678305 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a27790-5bca-4647-9122-15c642b45835 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:32.682847 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5846bfcd-24a3-4899-b5e0-53f8a7ddf240 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad258275cb964f34bf59f9360724ce95'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:32.690706 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a27790-5bca-4647-9122-15c642b45835 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:32.692534 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a27790-5bca-4647-9122-15c642b45835 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:32.695294 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5846bfcd-24a3-4899-b5e0-53f8a7ddf240 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad258275cb964f34bf59f9360724ce95'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:32.697974 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5846bfcd-24a3-4899-b5e0-53f8a7ddf240 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad258275cb964f34bf59f9360724ce95'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:32.705927 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1092/2204] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:20:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 05:20:32.720153 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5846bfcd-24a3-4899-b5e0-53f8a7ddf240 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad258275cb964f34bf59f9360724ce95'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:32.729728 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1113/2205] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:20:32 2022] DELETE /identity/v3/users/ad258275cb964f34bf59f9360724ce95 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:20:32.735433 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d91ea3a-b864-4d4b-b9c0-53368525393f None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:32.735827 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d91ea3a-b864-4d4b-b9c0-53368525393f None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:32.736590 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d91ea3a-b864-4d4b-b9c0-53368525393f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:32.761022 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-7d91ea3a-b864-4d4b-b9c0-53368525393f None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:32.762441 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-7d91ea3a-b864-4d4b-b9c0-53368525393f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:32.849066 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1093/2206] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:32.873625 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846bfcd-24a3-4899-b5e0-53f8a7ddf240 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:32.875533 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5846bfcd-24a3-4899-b5e0-53f8a7ddf240 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:32.953011 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846bfcd-24a3-4899-b5e0-53f8a7ddf240 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:32.956764 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043b9970-c1d5-46d6-9b0e-85da25f7571a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:32.957384 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b9970-c1d5-46d6-9b0e-85da25f7571a service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:32.957662 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b9970-c1d5-46d6-9b0e-85da25f7571a service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:32.957932 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b9970-c1d5-46d6-9b0e-85da25f7571a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:32.991166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043b9970-c1d5-46d6-9b0e-85da25f7571a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:32.998258 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043b9970-c1d5-46d6-9b0e-85da25f7571a service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:33.042493 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1114/2207] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:20:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 05:20:33.588269 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d91ea3a-b864-4d4b-b9c0-53368525393f None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:33.590521 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-7d91ea3a-b864-4d4b-b9c0-53368525393f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:33.620685 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d91ea3a-b864-4d4b-b9c0-53368525393f None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:33.626605 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73802ca-9890-4ac4-ab46-0185d3b37794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:33.627851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73802ca-9890-4ac4-ab46-0185d3b37794 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:33.628339 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73802ca-9890-4ac4-ab46-0185d3b37794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:33.628814 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73802ca-9890-4ac4-ab46-0185d3b37794 admin admin] PATH_INFO: `/v3/projects/4b93d7cc85a343729d23f81d492e9783` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:33.630768 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73802ca-9890-4ac4-ab46-0185d3b37794 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b93d7cc85a343729d23f81d492e9783)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:33.634988 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73802ca-9890-4ac4-ab46-0185d3b37794 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:33.707151 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:33.759887 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c73802ca-9890-4ac4-ab46-0185d3b37794 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b93d7cc85a343729d23f81d492e9783', 'request_id': 'req-c73802ca-9890-4ac4-ab46-0185d3b37794'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:33.771236 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c73802ca-9890-4ac4-ab46-0185d3b37794 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b93d7cc85a343729d23f81d492e9783', 'request_id': 'req-c73802ca-9890-4ac4-ab46-0185d3b37794'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:33.799109 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c73802ca-9890-4ac4-ab46-0185d3b37794 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b93d7cc85a343729d23f81d492e9783', 'request_id': 'req-c73802ca-9890-4ac4-ab46-0185d3b37794'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:33.855636 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c73802ca-9890-4ac4-ab46-0185d3b37794 admin admin] The token cache is being invalidate because project 4b93d7cc85a343729d23f81d492e9783 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:20:33.857069 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c73802ca-9890-4ac4-ab46-0185d3b37794 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:33.858417 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c73802ca-9890-4ac4-ab46-0185d3b37794 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:33.863155 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043b9970-c1d5-46d6-9b0e-85da25f7571a service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:33.864616 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-043b9970-c1d5-46d6-9b0e-85da25f7571a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:33.866372 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1094/2208] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:20:33 2022] DELETE /identity/v3/projects/4b93d7cc85a343729d23f81d492e9783 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:33.871136 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0f48c-8ea8-4739-8ad8-4fc3fe8984cf None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:33.871696 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0f48c-8ea8-4739-8ad8-4fc3fe8984cf None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:33.872155 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0f48c-8ea8-4739-8ad8-4fc3fe8984cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:33.896234 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043b9970-c1d5-46d6-9b0e-85da25f7571a service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:33.909609 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-69b0f48c-8ea8-4739-8ad8-4fc3fe8984cf None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:33.911067 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-69b0f48c-8ea8-4739-8ad8-4fc3fe8984cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:33.998071 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1095/2209] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:20:34.009722 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48acfc57-ab0b-4c05-986e-98849fa51565 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:34.010478 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48acfc57-ab0b-4c05-986e-98849fa51565 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:34.010969 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48acfc57-ab0b-4c05-986e-98849fa51565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:34.040585 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ebf1fe-790d-46c1-ae46-5325b86af241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:34.041368 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ebf1fe-790d-46c1-ae46-5325b86af241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:34.041801 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ebf1fe-790d-46c1-ae46-5325b86af241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:34.042199 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ebf1fe-790d-46c1-ae46-5325b86af241 admin admin] PATH_INFO: `/v3/projects/ce4e297be0b2485ebc2673b029067b5f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:34.044906 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ebf1fe-790d-46c1-ae46-5325b86af241 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce4e297be0b2485ebc2673b029067b5f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:34.049694 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ebf1fe-790d-46c1-ae46-5325b86af241 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:34.066055 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-48acfc57-ab0b-4c05-986e-98849fa51565 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:34.083441 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-48acfc57-ab0b-4c05-986e-98849fa51565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:34.108284 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1096/2210] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:34.109591 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48acfc57-ab0b-4c05-986e-98849fa51565 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:34.120424 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48acfc57-ab0b-4c05-986e-98849fa51565 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:34.135388 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:34.160511 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d2ebf1fe-790d-46c1-ae46-5325b86af241 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce4e297be0b2485ebc2673b029067b5f', 'request_id': 'req-d2ebf1fe-790d-46c1-ae46-5325b86af241'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:34.171012 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d2ebf1fe-790d-46c1-ae46-5325b86af241 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce4e297be0b2485ebc2673b029067b5f', 'request_id': 'req-d2ebf1fe-790d-46c1-ae46-5325b86af241'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:34.187354 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d2ebf1fe-790d-46c1-ae46-5325b86af241 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce4e297be0b2485ebc2673b029067b5f', 'request_id': 'req-d2ebf1fe-790d-46c1-ae46-5325b86af241'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:34.213138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d2ebf1fe-790d-46c1-ae46-5325b86af241 admin admin] The token cache is being invalidate because project ce4e297be0b2485ebc2673b029067b5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:20:34.213992 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d2ebf1fe-790d-46c1-ae46-5325b86af241 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:34.215898 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d2ebf1fe-790d-46c1-ae46-5325b86af241 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:34.218036 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1115/2211] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:20:33 2022] DELETE /identity/v3/projects/ce4e297be0b2485ebc2673b029067b5f => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:34.222570 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ebf1fe-790d-46c1-ae46-5325b86af241 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:34.224578 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d2ebf1fe-790d-46c1-ae46-5325b86af241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:34.231420 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668eab8b-7802-4709-8613-8721d9004ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:34.232146 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668eab8b-7802-4709-8613-8721d9004ced admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:34.232488 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668eab8b-7802-4709-8613-8721d9004ced admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:34.232851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668eab8b-7802-4709-8613-8721d9004ced admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:34.233606 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668eab8b-7802-4709-8613-8721d9004ced admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:20:34.233971 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668eab8b-7802-4709-8613-8721d9004ced admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:34.236359 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668eab8b-7802-4709-8613-8721d9004ced admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:34.248092 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1097/2212] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:20:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:20:34.252723 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668eab8b-7802-4709-8613-8721d9004ced admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:34.254654 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-668eab8b-7802-4709-8613-8721d9004ced admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:34.270553 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668eab8b-7802-4709-8613-8721d9004ced admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:34.276079 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001fefcc-e0dc-42ad-873a-e80f50ceb641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:34.276793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001fefcc-e0dc-42ad-873a-e80f50ceb641 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:34.277108 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001fefcc-e0dc-42ad-873a-e80f50ceb641 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:34.277363 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001fefcc-e0dc-42ad-873a-e80f50ceb641 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:34.278151 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001fefcc-e0dc-42ad-873a-e80f50ceb641 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:34.280168 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001fefcc-e0dc-42ad-873a-e80f50ceb641 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:34.293944 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ebf1fe-790d-46c1-ae46-5325b86af241 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:34.298738 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e8c895-8977-4c28-a46e-570bd123ba64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:34.299371 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8c895-8977-4c28-a46e-570bd123ba64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:34.299645 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8c895-8977-4c28-a46e-570bd123ba64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:34.299912 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8c895-8977-4c28-a46e-570bd123ba64 admin admin] PATH_INFO: `/v3/users/fae438d1761643f3b7f6d1023ce5f6a7` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:34.301046 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e8c895-8977-4c28-a46e-570bd123ba64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fae438d1761643f3b7f6d1023ce5f6a7)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:34.303059 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e8c895-8977-4c28-a46e-570bd123ba64 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:34.308890 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1098/2213] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:20:34 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:20:34.313529 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001fefcc-e0dc-42ad-873a-e80f50ceb641 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:34.341597 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:34.362496 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-76e8c895-8977-4c28-a46e-570bd123ba64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fae438d1761643f3b7f6d1023ce5f6a7'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:34.370284 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-76e8c895-8977-4c28-a46e-570bd123ba64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fae438d1761643f3b7f6d1023ce5f6a7'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:34.384101 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-76e8c895-8977-4c28-a46e-570bd123ba64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fae438d1761643f3b7f6d1023ce5f6a7'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:34.384987 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-76e8c895-8977-4c28-a46e-570bd123ba64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fae438d1761643f3b7f6d1023ce5f6a7'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:34.417804 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001fefcc-e0dc-42ad-873a-e80f50ceb641 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:34.419606 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-76e8c895-8977-4c28-a46e-570bd123ba64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fae438d1761643f3b7f6d1023ce5f6a7'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:34.437845 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1116/2214] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:20:34 2022] DELETE /identity/v3/users/fae438d1761643f3b7f6d1023ce5f6a7 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:34.438200 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa95fd-9945-4655-bf27-01a3aeb75589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:34.438200 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa95fd-9945-4655-bf27-01a3aeb75589 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:34.438200 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa95fd-9945-4655-bf27-01a3aeb75589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:34.438200 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa95fd-9945-4655-bf27-01a3aeb75589 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:34.439009 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aa95fd-9945-4655-bf27-01a3aeb75589 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:34.441274 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aa95fd-9945-4655-bf27-01a3aeb75589 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:34.489142 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1099/2215] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:20:34 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:20:34.491332 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e8c895-8977-4c28-a46e-570bd123ba64 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:34.493137 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-76e8c895-8977-4c28-a46e-570bd123ba64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:34.496665 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d4ae3-71ad-4014-8f2a-6f9d0e44a7c3 None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:34.497188 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d4ae3-71ad-4014-8f2a-6f9d0e44a7c3 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:34.497539 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d4ae3-71ad-4014-8f2a-6f9d0e44a7c3 None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:34.499542 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1100/2216] 10.222.0.36 () {54 vars in 921 bytes} [Wed Aug 31 05:20:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:20:34.534997 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d92a53-b747-4ab6-ba57-1e451311823e None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:34.535801 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d92a53-b747-4ab6-ba57-1e451311823e None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:34.536116 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d92a53-b747-4ab6-ba57-1e451311823e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:34.581385 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-f4d92a53-b747-4ab6-ba57-1e451311823e None None] MFA Rules not processed for user `bd4387035099416789009f0ecb8ad282`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:34.582753 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-f4d92a53-b747-4ab6-ba57-1e451311823e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:34.615682 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:34.632377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e8c895-8977-4c28-a46e-570bd123ba64 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:34.661296 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d2b99-5e87-4474-82b3-31dd676f2bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:34.661990 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d2b99-5e87-4474-82b3-31dd676f2bbc admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:34.662362 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d2b99-5e87-4474-82b3-31dd676f2bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:34.662640 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d2b99-5e87-4474-82b3-31dd676f2bbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:34.663361 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d2b99-5e87-4474-82b3-31dd676f2bbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:34.664946 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d2b99-5e87-4474-82b3-31dd676f2bbc admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:34.683232 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1101/2217] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Aug 31 05:20:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:20:34.684240 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d92a53-b747-4ab6-ba57-1e451311823e None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:34.685763 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-f4d92a53-b747-4ab6-ba57-1e451311823e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:34.695717 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1117/2218] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:34 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:20:34.699095 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d2b99-5e87-4474-82b3-31dd676f2bbc admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:34.712296 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d92a53-b747-4ab6-ba57-1e451311823e None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:34.718825 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d2b99-5e87-4474-82b3-31dd676f2bbc admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:34.719918 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e5a85-1a0e-4aef-847c-5f824b584717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:34.721010 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e5a85-1a0e-4aef-847c-5f824b584717 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:34.721444 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e5a85-1a0e-4aef-847c-5f824b584717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:34.722148 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e5a85-1a0e-4aef-847c-5f824b584717 admin admin] PATH_INFO: `/v3/users/69e6e2487fce4c03a8770443791c5b8a` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:34.723663 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2e5a85-1a0e-4aef-847c-5f824b584717 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69e6e2487fce4c03a8770443791c5b8a)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:34.726740 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38bece-3c0c-4075-a826-6c68036f292a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:34.727158 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2e5a85-1a0e-4aef-847c-5f824b584717 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:34.727483 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38bece-3c0c-4075-a826-6c68036f292a admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:34.728041 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38bece-3c0c-4075-a826-6c68036f292a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:34.728937 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38bece-3c0c-4075-a826-6c68036f292a admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:34.729670 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca38bece-3c0c-4075-a826-6c68036f292a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:34.731564 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca38bece-3c0c-4075-a826-6c68036f292a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:34.753798 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1118/2219] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:34 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:20:34.753798 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38bece-3c0c-4075-a826-6c68036f292a admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:34.753798 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ca38bece-3c0c-4075-a826-6c68036f292a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:34.778255 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38bece-3c0c-4075-a826-6c68036f292a admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:34.783806 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fd5bf8-b97a-4515-8040-8934abb7b6ac service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd4387035099416789009f0ecb8ad282', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:34.784446 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd5bf8-b97a-4515-8040-8934abb7b6ac service placement] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:34.784699 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd5bf8-b97a-4515-8040-8934abb7b6ac service placement] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:34.784961 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd5bf8-b97a-4515-8040-8934abb7b6ac service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:34.794933 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fd5bf8-b97a-4515-8040-8934abb7b6ac service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:34.796619 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fd5bf8-b97a-4515-8040-8934abb7b6ac service placement] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:34.804468 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9e2e5a85-1a0e-4aef-847c-5f824b584717 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69e6e2487fce4c03a8770443791c5b8a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:34.807417 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1119/2220] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:20:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:20:34.808283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fd5bf8-b97a-4515-8040-8934abb7b6ac service placement] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:34.812877 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9e2e5a85-1a0e-4aef-847c-5f824b584717 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69e6e2487fce4c03a8770443791c5b8a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:34.813812 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9e2e5a85-1a0e-4aef-847c-5f824b584717 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69e6e2487fce4c03a8770443791c5b8a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:34.821281 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fd5bf8-b97a-4515-8040-8934abb7b6ac service placement] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:34.824863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9e2e5a85-1a0e-4aef-847c-5f824b584717 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69e6e2487fce4c03a8770443791c5b8a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:34.826240 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-9e2e5a85-1a0e-4aef-847c-5f824b584717 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69e6e2487fce4c03a8770443791c5b8a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:34.829513 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeb896e-2244-422f-a78d-2c95b51e3be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:34.830694 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb896e-2244-422f-a78d-2c95b51e3be1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:34.830694 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb896e-2244-422f-a78d-2c95b51e3be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:34.830989 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb896e-2244-422f-a78d-2c95b51e3be1 admin admin] PATH_INFO: `/v3/projects/5350ea7b1c5b41b893627b862bfa8abc/users/a723a4ff5f3042ad97b098a1f3116a11/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:34.848381 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeb896e-2244-422f-a78d-2c95b51e3be1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5350ea7b1c5b41b893627b862bfa8abc, user_id=a723a4ff5f3042ad97b098a1f3116a11, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:34.850447 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1102/2221] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:20:34 2022] DELETE /identity/v3/users/69e6e2487fce4c03a8770443791c5b8a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:34.854322 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeb896e-2244-422f-a78d-2c95b51e3be1 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:34.860316 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e5a85-1a0e-4aef-847c-5f824b584717 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:34.862198 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-9e2e5a85-1a0e-4aef-847c-5f824b584717 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:34.876462 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1120/2222] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:20:34 2022] PUT /identity/v3/projects/5350ea7b1c5b41b893627b862bfa8abc/users/a723a4ff5f3042ad97b098a1f3116a11/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:34.887114 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728635a3-c00c-42f9-a047-4d78399f898a None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:34.887399 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728635a3-c00c-42f9-a047-4d78399f898a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:34.887668 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728635a3-c00c-42f9-a047-4d78399f898a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:34.890164 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:34.918574 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-728635a3-c00c-42f9-a047-4d78399f898a None None] MFA Rules not processed for user `a723a4ff5f3042ad97b098a1f3116a11`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:34.920126 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-728635a3-c00c-42f9-a047-4d78399f898a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:34.954046 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e5a85-1a0e-4aef-847c-5f824b584717 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:35.015285 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcc0bbd-89de-4474-af0a-0e7f096b1ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:35.016043 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc0bbd-89de-4474-af0a-0e7f096b1ef7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:35.016237 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc0bbd-89de-4474-af0a-0e7f096b1ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:35.016515 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc0bbd-89de-4474-af0a-0e7f096b1ef7 admin admin] PATH_INFO: `/v3/projects/a031059f951b4f1aa003a1ee51833e4d` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:35.017791 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcc0bbd-89de-4474-af0a-0e7f096b1ef7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a031059f951b4f1aa003a1ee51833e4d)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:35.018142 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1121/2223] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:20:35.020024 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcc0bbd-89de-4474-af0a-0e7f096b1ef7 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:35.031460 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728635a3-c00c-42f9-a047-4d78399f898a None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:35.033026 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-728635a3-c00c-42f9-a047-4d78399f898a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:35.080412 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-afcc0bbd-89de-4474-af0a-0e7f096b1ef7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a031059f951b4f1aa003a1ee51833e4d', 'request_id': 'req-afcc0bbd-89de-4474-af0a-0e7f096b1ef7'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:35.091351 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-afcc0bbd-89de-4474-af0a-0e7f096b1ef7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a031059f951b4f1aa003a1ee51833e4d', 'request_id': 'req-afcc0bbd-89de-4474-af0a-0e7f096b1ef7'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:35.099825 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728635a3-c00c-42f9-a047-4d78399f898a None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:35.100109 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-afcc0bbd-89de-4474-af0a-0e7f096b1ef7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a031059f951b4f1aa003a1ee51833e4d', 'request_id': 'req-afcc0bbd-89de-4474-af0a-0e7f096b1ef7'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:35.117648 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-afcc0bbd-89de-4474-af0a-0e7f096b1ef7 admin admin] The token cache is being invalidate because project a031059f951b4f1aa003a1ee51833e4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:20:35.117932 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-afcc0bbd-89de-4474-af0a-0e7f096b1ef7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:35.118631 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-afcc0bbd-89de-4474-af0a-0e7f096b1ef7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:35.120127 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1103/2224] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:20:34 2022] DELETE /identity/v3/projects/a031059f951b4f1aa003a1ee51833e4d => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:20:35.123132 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcc0bbd-89de-4474-af0a-0e7f096b1ef7 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:35.124873 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-afcc0bbd-89de-4474-af0a-0e7f096b1ef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:35.145043 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:35.155258 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb6e7f8-d4ec-40a2-9338-aec290bf2c5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:35.155900 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb6e7f8-d4ec-40a2-9338-aec290bf2c5b service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:35.156178 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb6e7f8-d4ec-40a2-9338-aec290bf2c5b service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:35.156454 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb6e7f8-d4ec-40a2-9338-aec290bf2c5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:35.158290 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6bb6e7f8-d4ec-40a2-9338-aec290bf2c5b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:35.187524 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:35.192666 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcc0bbd-89de-4474-af0a-0e7f096b1ef7 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:35.199743 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b0ebfe-3891-4cb3-8ec7-857fa7ce963d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:35.200443 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b0ebfe-3891-4cb3-8ec7-857fa7ce963d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:35.201175 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b0ebfe-3891-4cb3-8ec7-857fa7ce963d admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:35.201589 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b0ebfe-3891-4cb3-8ec7-857fa7ce963d admin admin] PATH_INFO: `/v3/projects/44a0d4c08d8044b1ab696dc757ba1a4d` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:35.202935 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b0ebfe-3891-4cb3-8ec7-857fa7ce963d admin admin] RBAC: Authorizing `identity:delete_project(project_id=44a0d4c08d8044b1ab696dc757ba1a4d)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:35.204880 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b0ebfe-3891-4cb3-8ec7-857fa7ce963d admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:35.218594 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb6e7f8-d4ec-40a2-9338-aec290bf2c5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:35.220023 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb6e7f8-d4ec-40a2-9338-aec290bf2c5b service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:35.228840 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1122/2225] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:20:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 05:20:35.229613 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb6e7f8-d4ec-40a2-9338-aec290bf2c5b service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:35.231086 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6bb6e7f8-d4ec-40a2-9338-aec290bf2c5b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:35.246046 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb6e7f8-d4ec-40a2-9338-aec290bf2c5b service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:35.267041 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-52b0ebfe-3891-4cb3-8ec7-857fa7ce963d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44a0d4c08d8044b1ab696dc757ba1a4d', 'request_id': 'req-52b0ebfe-3891-4cb3-8ec7-857fa7ce963d'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:35.279011 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-52b0ebfe-3891-4cb3-8ec7-857fa7ce963d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44a0d4c08d8044b1ab696dc757ba1a4d', 'request_id': 'req-52b0ebfe-3891-4cb3-8ec7-857fa7ce963d'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:35.287426 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-52b0ebfe-3891-4cb3-8ec7-857fa7ce963d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44a0d4c08d8044b1ab696dc757ba1a4d', 'request_id': 'req-52b0ebfe-3891-4cb3-8ec7-857fa7ce963d'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:35.303174 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-52b0ebfe-3891-4cb3-8ec7-857fa7ce963d admin admin] The token cache is being invalidate because project 44a0d4c08d8044b1ab696dc757ba1a4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:20:35.303541 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-52b0ebfe-3891-4cb3-8ec7-857fa7ce963d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:35.304248 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-52b0ebfe-3891-4cb3-8ec7-857fa7ce963d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:35.305963 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1104/2226] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:20:35 2022] DELETE /identity/v3/projects/44a0d4c08d8044b1ab696dc757ba1a4d => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:35.312389 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7bdb0a-ca19-4374-8543-0f4578c20297 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:35.313026 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7bdb0a-ca19-4374-8543-0f4578c20297 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:35.313525 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7bdb0a-ca19-4374-8543-0f4578c20297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:35.333747 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a6cbd7-12ef-4ecb-b885-6193396173cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:35.334348 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a6cbd7-12ef-4ecb-b885-6193396173cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:35.334628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a6cbd7-12ef-4ecb-b885-6193396173cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:35.334896 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a6cbd7-12ef-4ecb-b885-6193396173cb admin admin] PATH_INFO: `/v3/users/cd229338453346fdad8ee6673b49616e` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:35.336045 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a6cbd7-12ef-4ecb-b885-6193396173cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd229338453346fdad8ee6673b49616e)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:35.337993 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a6cbd7-12ef-4ecb-b885-6193396173cb admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:35.338913 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-fd7bdb0a-ca19-4374-8543-0f4578c20297 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:35.340984 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-fd7bdb0a-ca19-4374-8543-0f4578c20297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:35.357116 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1105/2227] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:20:35.358174 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f4387-2afb-4485-a0b0-fea1dfcb36fd None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:35.358598 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f4387-2afb-4485-a0b0-fea1dfcb36fd None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:35.358935 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f4387-2afb-4485-a0b0-fea1dfcb36fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:35.381109 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-b27f4387-2afb-4485-a0b0-fea1dfcb36fd None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:35.382513 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b27f4387-2afb-4485-a0b0-fea1dfcb36fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:35.394888 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1106/2228] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:20:35.396292 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f4387-2afb-4485-a0b0-fea1dfcb36fd None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:35.397208 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d9a6cbd7-12ef-4ecb-b885-6193396173cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd229338453346fdad8ee6673b49616e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:35.405690 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d9a6cbd7-12ef-4ecb-b885-6193396173cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd229338453346fdad8ee6673b49616e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:35.406984 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f4387-2afb-4485-a0b0-fea1dfcb36fd None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:35.413083 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3472bee4-39c1-4ef1-a85a-472d06c3df9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:35.413785 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472bee4-39c1-4ef1-a85a-472d06c3df9a admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:35.414117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472bee4-39c1-4ef1-a85a-472d06c3df9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:35.414424 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472bee4-39c1-4ef1-a85a-472d06c3df9a admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:35.414907 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d9a6cbd7-12ef-4ecb-b885-6193396173cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd229338453346fdad8ee6673b49616e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:35.415313 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3472bee4-39c1-4ef1-a85a-472d06c3df9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:20:35.415600 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d9a6cbd7-12ef-4ecb-b885-6193396173cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd229338453346fdad8ee6673b49616e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:35.415702 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3472bee4-39c1-4ef1-a85a-472d06c3df9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:35.417833 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3472bee4-39c1-4ef1-a85a-472d06c3df9a admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:35.430243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d9a6cbd7-12ef-4ecb-b885-6193396173cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd229338453346fdad8ee6673b49616e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:35.432186 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1107/2229] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:20:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:20:35.436841 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3472bee4-39c1-4ef1-a85a-472d06c3df9a admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:35.438101 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1123/2230] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:20:35 2022] DELETE /identity/v3/users/cd229338453346fdad8ee6673b49616e => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:35.438649 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3472bee4-39c1-4ef1-a85a-472d06c3df9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:35.452249 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3472bee4-39c1-4ef1-a85a-472d06c3df9a admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:35.559052 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cd57fa-1084-421b-ac60-f58fe148313d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:35.559670 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd57fa-1084-421b-ac60-f58fe148313d admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:35.560190 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd57fa-1084-421b-ac60-f58fe148313d admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:35.560190 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd57fa-1084-421b-ac60-f58fe148313d admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:35.560878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cd57fa-1084-421b-ac60-f58fe148313d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:35.563415 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cd57fa-1084-421b-ac60-f58fe148313d admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:35.605275 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1108/2231] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:20:35 2022] POST /identity/v3/projects => generated 391 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:20:35.606421 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a6cbd7-12ef-4ecb-b885-6193396173cb admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:35.620679 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a6cbd7-12ef-4ecb-b885-6193396173cb admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:35.638303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:35.697387 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2722c8c-258c-4a22-8062-5c762087b728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:35.698253 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2722c8c-258c-4a22-8062-5c762087b728 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:35.698559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2722c8c-258c-4a22-8062-5c762087b728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:35.698850 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2722c8c-258c-4a22-8062-5c762087b728 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:35.699640 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2722c8c-258c-4a22-8062-5c762087b728 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:35.701679 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2722c8c-258c-4a22-8062-5c762087b728 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:35.728157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1124/2232] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:20:35 2022] POST /identity/v3/users => generated 296 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:20:35.731369 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cd57fa-1084-421b-ac60-f58fe148313d admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:35.746912 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cd57fa-1084-421b-ac60-f58fe148313d admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:35.752915 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca8872-964c-4266-91e2-6f777dc50beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:35.753802 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca8872-964c-4266-91e2-6f777dc50beb admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:35.754428 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca8872-964c-4266-91e2-6f777dc50beb admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:35.754428 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca8872-964c-4266-91e2-6f777dc50beb admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:35.755148 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca8872-964c-4266-91e2-6f777dc50beb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:35.756940 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca8872-964c-4266-91e2-6f777dc50beb admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:35.772457 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1109/2233] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:20:35.773824 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2722c8c-258c-4a22-8062-5c762087b728 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:35.786355 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2722c8c-258c-4a22-8062-5c762087b728 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:35.792676 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a4ce6c-ac57-4a39-a0cb-5e4c4f0bac72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:35.793349 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a4ce6c-ac57-4a39-a0cb-5e4c4f0bac72 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:35.793719 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a4ce6c-ac57-4a39-a0cb-5e4c4f0bac72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:35.793883 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a4ce6c-ac57-4a39-a0cb-5e4c4f0bac72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:35.794700 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a4ce6c-ac57-4a39-a0cb-5e4c4f0bac72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:35.796194 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca8872-964c-4266-91e2-6f777dc50beb admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:35.796298 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a4ce6c-ac57-4a39-a0cb-5e4c4f0bac72 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:35.799168 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-2eca8872-964c-4266-91e2-6f777dc50beb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:35.818290 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca8872-964c-4266-91e2-6f777dc50beb admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:35.822722 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1125/2234] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:20:35.824065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a4ce6c-ac57-4a39-a0cb-5e4c4f0bac72 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:35.828588 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa59b5-44bd-4948-8dfc-c198b739171b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:35.834890 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa59b5-44bd-4948-8dfc-c198b739171b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:35.835228 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa59b5-44bd-4948-8dfc-c198b739171b admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:35.835668 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa59b5-44bd-4948-8dfc-c198b739171b admin admin] PATH_INFO: `/v3/projects/dfbd62d6cb104e7b9e1c9ed5dcacdebd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:35.837052 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aa59b5-44bd-4948-8dfc-c198b739171b admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfbd62d6cb104e7b9e1c9ed5dcacdebd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:35.837333 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a4ce6c-ac57-4a39-a0cb-5e4c4f0bac72 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:35.839370 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aa59b5-44bd-4948-8dfc-c198b739171b admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:35.843000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7a63c-a858-406e-a4ee-d8e170bf1af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:35.843672 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7a63c-a858-406e-a4ee-d8e170bf1af7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:35.843952 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7a63c-a858-406e-a4ee-d8e170bf1af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:35.844223 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7a63c-a858-406e-a4ee-d8e170bf1af7 admin admin] PATH_INFO: `/v3/projects/597fddc7874a4133987f79807674561f/users/806b011517ef4201a86c90472e0740d6/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:35.861488 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a7a63c-a858-406e-a4ee-d8e170bf1af7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=597fddc7874a4133987f79807674561f, user_id=806b011517ef4201a86c90472e0740d6, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:35.864251 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a7a63c-a858-406e-a4ee-d8e170bf1af7 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:35.875961 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:35.884107 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1126/2235] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:20:35 2022] PUT /identity/v3/projects/597fddc7874a4133987f79807674561f/users/806b011517ef4201a86c90472e0740d6/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:35.890661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e662f8-ff36-4a59-bbee-2379b03d518e None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:35.891026 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e662f8-ff36-4a59-bbee-2379b03d518e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:35.891368 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e662f8-ff36-4a59-bbee-2379b03d518e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:35.896807 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-64aa59b5-44bd-4948-8dfc-c198b739171b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfbd62d6cb104e7b9e1c9ed5dcacdebd', 'request_id': 'req-64aa59b5-44bd-4948-8dfc-c198b739171b'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:35.903845 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-64aa59b5-44bd-4948-8dfc-c198b739171b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfbd62d6cb104e7b9e1c9ed5dcacdebd', 'request_id': 'req-64aa59b5-44bd-4948-8dfc-c198b739171b'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:35.909931 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-64aa59b5-44bd-4948-8dfc-c198b739171b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfbd62d6cb104e7b9e1c9ed5dcacdebd', 'request_id': 'req-64aa59b5-44bd-4948-8dfc-c198b739171b'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:35.913977 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-31e662f8-ff36-4a59-bbee-2379b03d518e None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:35.915618 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-31e662f8-ff36-4a59-bbee-2379b03d518e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:35.924189 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-64aa59b5-44bd-4948-8dfc-c198b739171b admin admin] The token cache is being invalidate because project dfbd62d6cb104e7b9e1c9ed5dcacdebd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:20:35.924701 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-64aa59b5-44bd-4948-8dfc-c198b739171b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:35.925474 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-64aa59b5-44bd-4948-8dfc-c198b739171b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:20:35.927325 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1110/2236] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:20:35 2022] DELETE /identity/v3/projects/dfbd62d6cb104e7b9e1c9ed5dcacdebd => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:20:35.933302 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc24481-efc4-402c-91c5-fd7ab5d49f95 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:35.933593 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc24481-efc4-402c-91c5-fd7ab5d49f95 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:35.933870 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc24481-efc4-402c-91c5-fd7ab5d49f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:35.957361 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-6dc24481-efc4-402c-91c5-fd7ab5d49f95 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:35.958693 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-6dc24481-efc4-402c-91c5-fd7ab5d49f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:35.992046 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1127/2237] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:20:36.002102 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e662f8-ff36-4a59-bbee-2379b03d518e None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:36.003533 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-31e662f8-ff36-4a59-bbee-2379b03d518e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:36.022729 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1111/2238] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:20:36.032530 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24757f90-749c-45db-a725-049023f05e53 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:36.032886 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24757f90-749c-45db-a725-049023f05e53 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:36.033158 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24757f90-749c-45db-a725-049023f05e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:36.053624 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-24757f90-749c-45db-a725-049023f05e53 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:36.054872 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-24757f90-749c-45db-a725-049023f05e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:36.063137 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1112/2239] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:20:36.067363 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24757f90-749c-45db-a725-049023f05e53 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:36.075210 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e662f8-ff36-4a59-bbee-2379b03d518e None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:36.077030 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24757f90-749c-45db-a725-049023f05e53 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:36.079302 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d424c763-e3c9-4356-beae-df89eee5f770 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:36.079926 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424c763-e3c9-4356-beae-df89eee5f770 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:36.080208 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424c763-e3c9-4356-beae-df89eee5f770 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:36.080479 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424c763-e3c9-4356-beae-df89eee5f770 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:36.081697 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1576173-dcdc-46f8-9dd9-7070e7743173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:36.082296 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1576173-dcdc-46f8-9dd9-7070e7743173 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:36.082576 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1576173-dcdc-46f8-9dd9-7070e7743173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:36.082854 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1576173-dcdc-46f8-9dd9-7070e7743173 admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:36.083521 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1576173-dcdc-46f8-9dd9-7070e7743173 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:20:36.083822 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1576173-dcdc-46f8-9dd9-7070e7743173 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:36.085423 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1576173-dcdc-46f8-9dd9-7070e7743173 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:36.090259 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d424c763-e3c9-4356-beae-df89eee5f770 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:36.091634 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d424c763-e3c9-4356-beae-df89eee5f770 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:36.097730 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1113/2240] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:20:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:20:36.101682 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1128/2241] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:20:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 05:20:36.102402 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1576173-dcdc-46f8-9dd9-7070e7743173 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:36.113179 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1576173-dcdc-46f8-9dd9-7070e7743173 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:36.119347 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720b215-87b8-4cda-b5f8-014007d9882b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:36.119952 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720b215-87b8-4cda-b5f8-014007d9882b admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:36.120242 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720b215-87b8-4cda-b5f8-014007d9882b admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:36.120552 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720b215-87b8-4cda-b5f8-014007d9882b admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:36.121216 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7720b215-87b8-4cda-b5f8-014007d9882b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:36.123330 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7720b215-87b8-4cda-b5f8-014007d9882b admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:36.157478 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1114/2242] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:20:36 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:20:36.158771 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d424c763-e3c9-4356-beae-df89eee5f770 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:36.169114 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d424c763-e3c9-4356-beae-df89eee5f770 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:36.179073 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:36.233617 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f560d48f-6e84-44b2-bd41-3400bb441f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:36.234697 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560d48f-6e84-44b2-bd41-3400bb441f39 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:36.235004 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560d48f-6e84-44b2-bd41-3400bb441f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:36.235385 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560d48f-6e84-44b2-bd41-3400bb441f39 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:36.236027 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f560d48f-6e84-44b2-bd41-3400bb441f39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:36.238017 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f560d48f-6e84-44b2-bd41-3400bb441f39 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:36.259809 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1129/2243] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:20:36 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:20:36.264162 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720b215-87b8-4cda-b5f8-014007d9882b admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:36.274921 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720b215-87b8-4cda-b5f8-014007d9882b admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:36.279742 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5298163f-809d-4f40-8b79-0d61dcb87188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:36.280425 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5298163f-809d-4f40-8b79-0d61dcb87188 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:36.280774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5298163f-809d-4f40-8b79-0d61dcb87188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:36.281051 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5298163f-809d-4f40-8b79-0d61dcb87188 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:36.281820 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5298163f-809d-4f40-8b79-0d61dcb87188 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:36.283564 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5298163f-809d-4f40-8b79-0d61dcb87188 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:36.297637 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1115/2244] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:20:36.298773 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f560d48f-6e84-44b2-bd41-3400bb441f39 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:36.310992 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f560d48f-6e84-44b2-bd41-3400bb441f39 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:36.317152 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f964bfac-c77b-4e2c-91e5-db6f4ae01816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:36.317758 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964bfac-c77b-4e2c-91e5-db6f4ae01816 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:36.318134 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964bfac-c77b-4e2c-91e5-db6f4ae01816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:36.318398 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964bfac-c77b-4e2c-91e5-db6f4ae01816 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:36.319042 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f964bfac-c77b-4e2c-91e5-db6f4ae01816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:36.320633 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f964bfac-c77b-4e2c-91e5-db6f4ae01816 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:36.335151 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1130/2245] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 05:20:36.337454 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5298163f-809d-4f40-8b79-0d61dcb87188 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:36.351884 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5298163f-809d-4f40-8b79-0d61dcb87188 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:36.358801 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f7e655-1ea3-4578-b11d-a9ede34f72da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:36.360116 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7e655-1ea3-4578-b11d-a9ede34f72da admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:36.360486 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7e655-1ea3-4578-b11d-a9ede34f72da admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:36.360855 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7e655-1ea3-4578-b11d-a9ede34f72da admin admin] PATH_INFO: `/v3/projects/6c40b2e1ebd54a4198b60ef3e3ec1d91/users/ae4f9098a0a8434d9bd8d495c3686594/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:36.378876 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f7e655-1ea3-4578-b11d-a9ede34f72da admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c40b2e1ebd54a4198b60ef3e3ec1d91, user_id=ae4f9098a0a8434d9bd8d495c3686594, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:36.381636 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f7e655-1ea3-4578-b11d-a9ede34f72da admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:36.395218 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1116/2246] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:20:36 2022] PUT /identity/v3/projects/6c40b2e1ebd54a4198b60ef3e3ec1d91/users/ae4f9098a0a8434d9bd8d495c3686594/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:36.400661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9096d-75ab-4606-be5b-89327b52a5b2 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:36.400941 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9096d-75ab-4606-be5b-89327b52a5b2 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:36.401212 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9096d-75ab-4606-be5b-89327b52a5b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:36.420550 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-8be9096d-75ab-4606-be5b-89327b52a5b2 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:36.421769 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8be9096d-75ab-4606-be5b-89327b52a5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:36.490442 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1131/2247] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:20:36.500512 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f7e655-1ea3-4578-b11d-a9ede34f72da admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:36.546278 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:36.569518 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f7e655-1ea3-4578-b11d-a9ede34f72da admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:36.573692 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ac5998-a1cf-402d-bd08-fc4391401dae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:36.574593 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ac5998-a1cf-402d-bd08-fc4391401dae service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:36.575687 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ac5998-a1cf-402d-bd08-fc4391401dae service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:36.575687 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ac5998-a1cf-402d-bd08-fc4391401dae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:36.587310 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ac5998-a1cf-402d-bd08-fc4391401dae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:36.591786 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ac5998-a1cf-402d-bd08-fc4391401dae service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:36.604469 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1117/2248] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:20:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:20:38.415626 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291e918-0d05-4956-aaf1-0f224889d958 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:38.416377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291e918-0d05-4956-aaf1-0f224889d958 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:38.416658 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291e918-0d05-4956-aaf1-0f224889d958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:38.504073 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-4291e918-0d05-4956-aaf1-0f224889d958 None None] MFA Rules not processed for user `ae4f9098a0a8434d9bd8d495c3686594`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:38.505306 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4291e918-0d05-4956-aaf1-0f224889d958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:38.693377 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ac5998-a1cf-402d-bd08-fc4391401dae service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:38.700786 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1132/2249] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:38.706459 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ac5998-a1cf-402d-bd08-fc4391401dae service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:38.717014 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c314a7-3691-4cc1-ba34-91e35165ba62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:38.717746 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c314a7-3691-4cc1-ba34-91e35165ba62 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:38.718065 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c314a7-3691-4cc1-ba34-91e35165ba62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:38.719553 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c314a7-3691-4cc1-ba34-91e35165ba62 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:38.721463 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c314a7-3691-4cc1-ba34-91e35165ba62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:38.726903 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c314a7-3691-4cc1-ba34-91e35165ba62 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:38.791500 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1118/2250] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:20:38 2022] POST /identity/v3/projects => generated 371 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:20:38.795967 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291e918-0d05-4956-aaf1-0f224889d958 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:38.810313 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291e918-0d05-4956-aaf1-0f224889d958 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:38.839482 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:38.910536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b37eef-7535-42d1-9798-b2472502009b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:38.911519 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b37eef-7535-42d1-9798-b2472502009b admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:38.912569 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b37eef-7535-42d1-9798-b2472502009b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:38.912990 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b37eef-7535-42d1-9798-b2472502009b admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:38.913804 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b37eef-7535-42d1-9798-b2472502009b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:38.916971 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b37eef-7535-42d1-9798-b2472502009b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:38.950086 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1133/2251] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:20:38 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:20:38.954238 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c314a7-3691-4cc1-ba34-91e35165ba62 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:38.968214 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c314a7-3691-4cc1-ba34-91e35165ba62 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:38.975221 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0958317a-efd2-4a18-8924-524c5139aac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:38.976072 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0958317a-efd2-4a18-8924-524c5139aac5 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:38.976072 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0958317a-efd2-4a18-8924-524c5139aac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:38.976357 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0958317a-efd2-4a18-8924-524c5139aac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:38.977013 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0958317a-efd2-4a18-8924-524c5139aac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:38.979265 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0958317a-efd2-4a18-8924-524c5139aac5 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:38.997371 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1119/2252] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:38 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:20:38.999634 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b37eef-7535-42d1-9798-b2472502009b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:39.018539 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b37eef-7535-42d1-9798-b2472502009b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:39.026789 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dd4f6f-fd27-4467-b714-31616610b794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:39.027541 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd4f6f-fd27-4467-b714-31616610b794 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:39.027820 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd4f6f-fd27-4467-b714-31616610b794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:39.028088 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd4f6f-fd27-4467-b714-31616610b794 admin admin] PATH_INFO: `/v3/projects/033bafebbcd04c5a889ded54a1767cdd/users/40968db1a35c4e55b002a949ede63cd9/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:39.056087 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dd4f6f-fd27-4467-b714-31616610b794 admin admin] RBAC: Authorizing `identity:create_grant(project_id=033bafebbcd04c5a889ded54a1767cdd, user_id=40968db1a35c4e55b002a949ede63cd9, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:39.059003 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dd4f6f-fd27-4467-b714-31616610b794 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:39.085111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1134/2253] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:20:38 2022] PUT /identity/v3/projects/033bafebbcd04c5a889ded54a1767cdd/users/40968db1a35c4e55b002a949ede63cd9/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:39.086428 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0958317a-efd2-4a18-8924-524c5139aac5 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:39.097304 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0958317a-efd2-4a18-8924-524c5139aac5 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:39.170783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e240b-3865-41e4-928f-e00589edeea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:39.171569 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e240b-3865-41e4-928f-e00589edeea9 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:39.171855 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e240b-3865-41e4-928f-e00589edeea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:39.172133 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e240b-3865-41e4-928f-e00589edeea9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:39.172792 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e240b-3865-41e4-928f-e00589edeea9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:39.174461 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e240b-3865-41e4-928f-e00589edeea9 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:39.187399 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1120/2254] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:20:39 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:20:39.191913 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dd4f6f-fd27-4467-b714-31616610b794 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:39.204056 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dd4f6f-fd27-4467-b714-31616610b794 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:39.209110 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f750d606-3625-488f-b5e9-a40b4b9a7c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:39.210068 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f750d606-3625-488f-b5e9-a40b4b9a7c48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:39.210375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f750d606-3625-488f-b5e9-a40b4b9a7c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:39.210686 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f750d606-3625-488f-b5e9-a40b4b9a7c48 admin admin] PATH_INFO: `/v3/projects/033bafebbcd04c5a889ded54a1767cdd/users/40968db1a35c4e55b002a949ede63cd9/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:39.213938 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f750d606-3625-488f-b5e9-a40b4b9a7c48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=033bafebbcd04c5a889ded54a1767cdd, user_id=40968db1a35c4e55b002a949ede63cd9, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:39.215999 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f750d606-3625-488f-b5e9-a40b4b9a7c48 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:39.228244 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1135/2255] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:20:39 2022] PUT /identity/v3/projects/033bafebbcd04c5a889ded54a1767cdd/users/40968db1a35c4e55b002a949ede63cd9/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:20:40.245390 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c47df-5e81-4504-ac14-37a9edfe824e None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:40.246077 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c47df-5e81-4504-ac14-37a9edfe824e None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:40.246077 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c47df-5e81-4504-ac14-37a9edfe824e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:40.274829 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-ae1c47df-5e81-4504-ac14-37a9edfe824e None None] MFA Rules not processed for user `40968db1a35c4e55b002a949ede63cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:40.276536 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-ae1c47df-5e81-4504-ac14-37a9edfe824e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:40.362603 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1121/2256] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:40 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:40.375076 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f750d606-3625-488f-b5e9-a40b4b9a7c48 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:40.376646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f750d606-3625-488f-b5e9-a40b4b9a7c48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:40.475816 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f750d606-3625-488f-b5e9-a40b4b9a7c48 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:40.485107 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac6d3b7-b189-40a4-afb9-c2f81d49a351 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:40.486346 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6d3b7-b189-40a4-afb9-c2f81d49a351 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:40.486786 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6d3b7-b189-40a4-afb9-c2f81d49a351 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:40.487227 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6d3b7-b189-40a4-afb9-c2f81d49a351 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:40.570705 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac6d3b7-b189-40a4-afb9-c2f81d49a351 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:40.572456 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac6d3b7-b189-40a4-afb9-c2f81d49a351 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:40.601620 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1136/2257] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:20:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:20:41.681002 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63c7a1-5ed6-4ff7-b289-063fc979dfe0 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:41.681473 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63c7a1-5ed6-4ff7-b289-063fc979dfe0 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:41.681718 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63c7a1-5ed6-4ff7-b289-063fc979dfe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:41.712025 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-be63c7a1-5ed6-4ff7-b289-063fc979dfe0 None None] MFA Rules not processed for user `806b011517ef4201a86c90472e0740d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:41.713236 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-be63c7a1-5ed6-4ff7-b289-063fc979dfe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:41.735649 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:41.791213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1122/2258] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:41 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:41.801220 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9971b-f641-4dcd-92a7-a86c52986ed8 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:41.801777 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9971b-f641-4dcd-92a7-a86c52986ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:41.801777 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9971b-f641-4dcd-92a7-a86c52986ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:41.823226 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d4e9971b-f641-4dcd-92a7-a86c52986ed8 None None] MFA Rules not processed for user `806b011517ef4201a86c90472e0740d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:41.824513 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d4e9971b-f641-4dcd-92a7-a86c52986ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:41.836846 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1137/2259] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:41 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:41.843117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be63c7a1-5ed6-4ff7-b289-063fc979dfe0 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:41.844705 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-be63c7a1-5ed6-4ff7-b289-063fc979dfe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:41.923991 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be63c7a1-5ed6-4ff7-b289-063fc979dfe0 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:41.929956 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bf7c61-ba41-493f-a324-289137906faa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:41.930644 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf7c61-ba41-493f-a324-289137906faa service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:41.930927 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf7c61-ba41-493f-a324-289137906faa service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:41.931198 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf7c61-ba41-493f-a324-289137906faa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:41.940031 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bf7c61-ba41-493f-a324-289137906faa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:41.941659 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bf7c61-ba41-493f-a324-289137906faa service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:41.951353 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1123/2260] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:20:41 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:20:42.836411 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b231a-ee67-49f3-a09a-dc69455ee9fd None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:42.836889 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b231a-ee67-49f3-a09a-dc69455ee9fd None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:42.837003 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b231a-ee67-49f3-a09a-dc69455ee9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:42.864474 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-cd7b231a-ee67-49f3-a09a-dc69455ee9fd None None] MFA Rules not processed for user `806b011517ef4201a86c90472e0740d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:20:42.865984 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-cd7b231a-ee67-49f3-a09a-dc69455ee9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:42.873448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1138/2261] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:20:42 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:20:45.267528 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bf7c61-ba41-493f-a324-289137906faa service nova] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:20:45.269121 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-86bf7c61-ba41-493f-a324-289137906faa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:20:45.285542 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bf7c61-ba41-493f-a324-289137906faa service nova] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:20:45.292265 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c429d39-6b7c-49fa-a948-29bccd784657 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:20:45.293432 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c429d39-6b7c-49fa-a948-29bccd784657 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:20:45.293928 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c429d39-6b7c-49fa-a948-29bccd784657 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:20:45.294430 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c429d39-6b7c-49fa-a948-29bccd784657 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:20:45.304743 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c429d39-6b7c-49fa-a948-29bccd784657 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:20:45.306416 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c429d39-6b7c-49fa-a948-29bccd784657 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:20:45.317802 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1124/2262] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:20:45 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:20:52.855722 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=88,keystone:INSERT=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:20:55.313296 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=58 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:21:21.374104 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b231a-ee67-49f3-a09a-dc69455ee9fd None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:21.375626 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-cd7b231a-ee67-49f3-a09a-dc69455ee9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:21:21.444959 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b231a-ee67-49f3-a09a-dc69455ee9fd None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:21.449632 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da235c-b17c-403a-9a0b-c916826ebae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:21.450315 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da235c-b17c-403a-9a0b-c916826ebae2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:21.450595 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da235c-b17c-403a-9a0b-c916826ebae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:21.450863 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da235c-b17c-403a-9a0b-c916826ebae2 admin admin] PATH_INFO: `/v3/users/a723a4ff5f3042ad97b098a1f3116a11` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:21.451973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da235c-b17c-403a-9a0b-c916826ebae2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a723a4ff5f3042ad97b098a1f3116a11)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:21.453907 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da235c-b17c-403a-9a0b-c916826ebae2 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:21.501680 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-50da235c-b17c-403a-9a0b-c916826ebae2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a723a4ff5f3042ad97b098a1f3116a11'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:21.508542 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-50da235c-b17c-403a-9a0b-c916826ebae2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a723a4ff5f3042ad97b098a1f3116a11'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:21.518509 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-50da235c-b17c-403a-9a0b-c916826ebae2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a723a4ff5f3042ad97b098a1f3116a11'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:21.519126 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-50da235c-b17c-403a-9a0b-c916826ebae2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a723a4ff5f3042ad97b098a1f3116a11'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:21.532612 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-50da235c-b17c-403a-9a0b-c916826ebae2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a723a4ff5f3042ad97b098a1f3116a11'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:21.539014 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1139/2263] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:21:21 2022] DELETE /identity/v3/users/a723a4ff5f3042ad97b098a1f3116a11 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:21:21.543848 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a570c-20d1-452e-9846-216947eac732 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:21.544382 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a570c-20d1-452e-9846-216947eac732 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:21.544382 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a570c-20d1-452e-9846-216947eac732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:21.563825 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-241a570c-20d1-452e-9846-216947eac732 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:21:21.565097 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-241a570c-20d1-452e-9846-216947eac732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:21:21.630780 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1125/2264] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:21:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:21:21.640568 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da235c-b17c-403a-9a0b-c916826ebae2 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:21.641979 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-50da235c-b17c-403a-9a0b-c916826ebae2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:21:21.702601 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da235c-b17c-403a-9a0b-c916826ebae2 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:21.706547 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db5722d-ac36-4afb-9fac-0e221c678db7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:21.707264 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5722d-ac36-4afb-9fac-0e221c678db7 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:21.707545 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5722d-ac36-4afb-9fac-0e221c678db7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:21.707811 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5722d-ac36-4afb-9fac-0e221c678db7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:21.716977 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db5722d-ac36-4afb-9fac-0e221c678db7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:21.718441 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db5722d-ac36-4afb-9fac-0e221c678db7 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:21.728105 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1140/2265] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:21:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:21:21.801129 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a570c-20d1-452e-9846-216947eac732 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:21.802876 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-241a570c-20d1-452e-9846-216947eac732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:21:21.816087 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a570c-20d1-452e-9846-216947eac732 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:21.821178 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369e963-dd83-4b84-b904-ceaf0816000a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:21.821832 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369e963-dd83-4b84-b904-ceaf0816000a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:21.822129 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369e963-dd83-4b84-b904-ceaf0816000a admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:21.822403 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369e963-dd83-4b84-b904-ceaf0816000a admin admin] PATH_INFO: `/v3/users/ba12eead104343819c2421428d35ea56` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:21.823638 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a369e963-dd83-4b84-b904-ceaf0816000a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba12eead104343819c2421428d35ea56)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:21.825597 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a369e963-dd83-4b84-b904-ceaf0816000a admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:21.883899 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a369e963-dd83-4b84-b904-ceaf0816000a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba12eead104343819c2421428d35ea56'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:21.891854 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a369e963-dd83-4b84-b904-ceaf0816000a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba12eead104343819c2421428d35ea56'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:21.892665 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a369e963-dd83-4b84-b904-ceaf0816000a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba12eead104343819c2421428d35ea56'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:21.902406 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a369e963-dd83-4b84-b904-ceaf0816000a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba12eead104343819c2421428d35ea56'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:21.903226 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a369e963-dd83-4b84-b904-ceaf0816000a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba12eead104343819c2421428d35ea56'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:21.920983 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1126/2266] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:21:21 2022] DELETE /identity/v3/users/ba12eead104343819c2421428d35ea56 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:21:21.925743 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389cc1e3-13d1-4dd1-aeb5-f59be0cc7bae None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:21.926051 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389cc1e3-13d1-4dd1-aeb5-f59be0cc7bae None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:21.926330 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389cc1e3-13d1-4dd1-aeb5-f59be0cc7bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:21.947876 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-389cc1e3-13d1-4dd1-aeb5-f59be0cc7bae None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:21:21.949290 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-389cc1e3-13d1-4dd1-aeb5-f59be0cc7bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:21:21.968707 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:21:22.027055 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369e963-dd83-4b84-b904-ceaf0816000a admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:22.031472 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a369e963-dd83-4b84-b904-ceaf0816000a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:21:22.038105 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1141/2267] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:21:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:21:22.046259 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369e963-dd83-4b84-b904-ceaf0816000a admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:22.051658 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389cc1e3-13d1-4dd1-aeb5-f59be0cc7bae None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:22.053299 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-389cc1e3-13d1-4dd1-aeb5-f59be0cc7bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:21:22.059662 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea130f0-3cb6-489b-ac67-0b4cb5cd35cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:22.060510 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea130f0-3cb6-489b-ac67-0b4cb5cd35cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:22.060626 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea130f0-3cb6-489b-ac67-0b4cb5cd35cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:22.060851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea130f0-3cb6-489b-ac67-0b4cb5cd35cd admin admin] PATH_INFO: `/v3/projects/5350ea7b1c5b41b893627b862bfa8abc` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:22.062739 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea130f0-3cb6-489b-ac67-0b4cb5cd35cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=5350ea7b1c5b41b893627b862bfa8abc)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:22.065806 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea130f0-3cb6-489b-ac67-0b4cb5cd35cd admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:22.145429 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-1ea130f0-3cb6-489b-ac67-0b4cb5cd35cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5350ea7b1c5b41b893627b862bfa8abc', 'request_id': 'req-1ea130f0-3cb6-489b-ac67-0b4cb5cd35cd'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:22.147188 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389cc1e3-13d1-4dd1-aeb5-f59be0cc7bae None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:22.156419 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-1ea130f0-3cb6-489b-ac67-0b4cb5cd35cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5350ea7b1c5b41b893627b862bfa8abc', 'request_id': 'req-1ea130f0-3cb6-489b-ac67-0b4cb5cd35cd'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:22.167302 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-1ea130f0-3cb6-489b-ac67-0b4cb5cd35cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5350ea7b1c5b41b893627b862bfa8abc', 'request_id': 'req-1ea130f0-3cb6-489b-ac67-0b4cb5cd35cd'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:22.172114 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:21:22.193523 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-1ea130f0-3cb6-489b-ac67-0b4cb5cd35cd admin admin] The token cache is being invalidate because project 5350ea7b1c5b41b893627b862bfa8abc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:21:22.193898 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-1ea130f0-3cb6-489b-ac67-0b4cb5cd35cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:22.194522 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-1ea130f0-3cb6-489b-ac67-0b4cb5cd35cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:22.197121 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1127/2268] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:21:22 2022] DELETE /identity/v3/projects/5350ea7b1c5b41b893627b862bfa8abc => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:21:22.229661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e551d43d-b1fe-42d7-b138-92d27d4454cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:22.244317 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e551d43d-b1fe-42d7-b138-92d27d4454cf service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:22.245218 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e551d43d-b1fe-42d7-b138-92d27d4454cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:22.245772 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e551d43d-b1fe-42d7-b138-92d27d4454cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:22.251883 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e551d43d-b1fe-42d7-b138-92d27d4454cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:21:22.340151 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e551d43d-b1fe-42d7-b138-92d27d4454cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:22.342288 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e551d43d-b1fe-42d7-b138-92d27d4454cf service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:22.356466 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1142/2269] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:21:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 05:21:22.647020 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea130f0-3cb6-489b-ac67-0b4cb5cd35cd admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:22.648687 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-1ea130f0-3cb6-489b-ac67-0b4cb5cd35cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:21:22.664085 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea130f0-3cb6-489b-ac67-0b4cb5cd35cd admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:22.668952 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713458d-ef8a-484d-befa-bc7e1e69b571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:22.669608 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713458d-ef8a-484d-befa-bc7e1e69b571 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:22.669880 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713458d-ef8a-484d-befa-bc7e1e69b571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:22.670188 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713458d-ef8a-484d-befa-bc7e1e69b571 admin admin] PATH_INFO: `/v3/projects/40fec664b405481abeab6c40c31c5ef8` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:22.671283 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c713458d-ef8a-484d-befa-bc7e1e69b571 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40fec664b405481abeab6c40c31c5ef8)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:22.673210 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c713458d-ef8a-484d-befa-bc7e1e69b571 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:22.722981 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c713458d-ef8a-484d-befa-bc7e1e69b571 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40fec664b405481abeab6c40c31c5ef8', 'request_id': 'req-c713458d-ef8a-484d-befa-bc7e1e69b571'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:22.729223 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c713458d-ef8a-484d-befa-bc7e1e69b571 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40fec664b405481abeab6c40c31c5ef8', 'request_id': 'req-c713458d-ef8a-484d-befa-bc7e1e69b571'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:22.735087 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c713458d-ef8a-484d-befa-bc7e1e69b571 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40fec664b405481abeab6c40c31c5ef8', 'request_id': 'req-c713458d-ef8a-484d-befa-bc7e1e69b571'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:22.749271 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c713458d-ef8a-484d-befa-bc7e1e69b571 admin admin] The token cache is being invalidate because project 40fec664b405481abeab6c40c31c5ef8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:21:22.749542 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c713458d-ef8a-484d-befa-bc7e1e69b571 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:22.750208 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c713458d-ef8a-484d-befa-bc7e1e69b571 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:22.751780 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1128/2270] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:21:22 2022] DELETE /identity/v3/projects/40fec664b405481abeab6c40c31c5ef8 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:21:22.757127 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b711b9-f0b5-470b-9403-99280056b2c7 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:22.757639 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b711b9-f0b5-470b-9403-99280056b2c7 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:22.757774 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b711b9-f0b5-470b-9403-99280056b2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:22.777832 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e9b711b9-f0b5-470b-9403-99280056b2c7 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:21:22.779169 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e9b711b9-f0b5-470b-9403-99280056b2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:21:22.805227 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:21:22.844570 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1143/2271] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:21:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:21:22.852741 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a3da7-ccdb-4d86-ad5f-88f80df74f22 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:22.853000 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a3da7-ccdb-4d86-ad5f-88f80df74f22 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:22.853332 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a3da7-ccdb-4d86-ad5f-88f80df74f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:22.873901 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-a11a3da7-ccdb-4d86-ad5f-88f80df74f22 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:21:22.875232 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a11a3da7-ccdb-4d86-ad5f-88f80df74f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:21:22.883960 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1129/2272] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:21:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:21:22.888407 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b711b9-f0b5-470b-9403-99280056b2c7 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:22.898679 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b711b9-f0b5-470b-9403-99280056b2c7 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:22.903690 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f3de7a-121d-4d06-aa43-a9721cdac2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:22.904305 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3de7a-121d-4d06-aa43-a9721cdac2d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:22.904581 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3de7a-121d-4d06-aa43-a9721cdac2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:22.904852 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3de7a-121d-4d06-aa43-a9721cdac2d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:22.905536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f3de7a-121d-4d06-aa43-a9721cdac2d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:21:22.905838 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f3de7a-121d-4d06-aa43-a9721cdac2d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:22.907489 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f3de7a-121d-4d06-aa43-a9721cdac2d5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:22.919745 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1144/2273] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:21:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:21:22.924183 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a3da7-ccdb-4d86-ad5f-88f80df74f22 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:22.933527 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a3da7-ccdb-4d86-ad5f-88f80df74f22 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:22.938689 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a2d309-3545-433d-8615-b0cc5a45799e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:22.939277 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2d309-3545-433d-8615-b0cc5a45799e admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:22.939550 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2d309-3545-433d-8615-b0cc5a45799e admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:22.939812 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2d309-3545-433d-8615-b0cc5a45799e admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:22.940425 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a2d309-3545-433d-8615-b0cc5a45799e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:22.942435 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a2d309-3545-433d-8615-b0cc5a45799e admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:22.980460 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1130/2274] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:21:22 2022] POST /identity/v3/projects => generated 369 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:21:22.985065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f3de7a-121d-4d06-aa43-a9721cdac2d5 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:22.994959 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f3de7a-121d-4d06-aa43-a9721cdac2d5 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:23.054450 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43929ffd-d8ce-4c67-abcb-3f1c10287724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:23.054995 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43929ffd-d8ce-4c67-abcb-3f1c10287724 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:23.055273 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43929ffd-d8ce-4c67-abcb-3f1c10287724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:23.055545 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43929ffd-d8ce-4c67-abcb-3f1c10287724 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:23.056204 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43929ffd-d8ce-4c67-abcb-3f1c10287724 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:23.058446 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43929ffd-d8ce-4c67-abcb-3f1c10287724 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:23.081800 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1145/2275] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:21:22 2022] POST /identity/v3/users => generated 285 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:21:23.086098 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a2d309-3545-433d-8615-b0cc5a45799e admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:23.096814 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a2d309-3545-433d-8615-b0cc5a45799e admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:23.101874 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e22920-e197-4453-874c-5dc61a61c557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:23.102496 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e22920-e197-4453-874c-5dc61a61c557 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:23.102780 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e22920-e197-4453-874c-5dc61a61c557 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:23.103103 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e22920-e197-4453-874c-5dc61a61c557 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:23.103778 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e22920-e197-4453-874c-5dc61a61c557 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:23.105385 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e22920-e197-4453-874c-5dc61a61c557 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:23.116116 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1131/2276] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:21:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:21:23.120154 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43929ffd-d8ce-4c67-abcb-3f1c10287724 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:23.130365 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43929ffd-d8ce-4c67-abcb-3f1c10287724 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:23.136093 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b45d0b6-53fc-4a8c-8d79-5a5dd4564045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:23.136934 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45d0b6-53fc-4a8c-8d79-5a5dd4564045 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:23.137205 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45d0b6-53fc-4a8c-8d79-5a5dd4564045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:23.137462 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45d0b6-53fc-4a8c-8d79-5a5dd4564045 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:23.138180 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b45d0b6-53fc-4a8c-8d79-5a5dd4564045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:23.139767 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b45d0b6-53fc-4a8c-8d79-5a5dd4564045 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:23.150593 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1146/2277] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:21:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:21:23.154824 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e22920-e197-4453-874c-5dc61a61c557 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:23.165218 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e22920-e197-4453-874c-5dc61a61c557 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:23.171194 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f41c4-854e-455b-b3f6-f14b25447708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:23.171824 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f41c4-854e-455b-b3f6-f14b25447708 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:23.172125 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f41c4-854e-455b-b3f6-f14b25447708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:23.172419 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f41c4-854e-455b-b3f6-f14b25447708 admin admin] PATH_INFO: `/v3/projects/fca7e32ef9214c44a34f72790208a668/users/20cf2265f6dd47678ee70ecb2d84a15a/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:23.188778 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452f41c4-854e-455b-b3f6-f14b25447708 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fca7e32ef9214c44a34f72790208a668, user_id=20cf2265f6dd47678ee70ecb2d84a15a, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:23.191513 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452f41c4-854e-455b-b3f6-f14b25447708 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:23.207271 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1132/2278] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:21:23 2022] PUT /identity/v3/projects/fca7e32ef9214c44a34f72790208a668/users/20cf2265f6dd47678ee70ecb2d84a15a/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:21:23.218894 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ed3e5-c4e6-4dcd-8535-4446de225e3d None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:23.219406 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ed3e5-c4e6-4dcd-8535-4446de225e3d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:23.219738 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ed3e5-c4e6-4dcd-8535-4446de225e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:23.268222 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d34ed3e5-c4e6-4dcd-8535-4446de225e3d None None] MFA Rules not processed for user `20cf2265f6dd47678ee70ecb2d84a15a`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:21:23.269513 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d34ed3e5-c4e6-4dcd-8535-4446de225e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:21:23.375173 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1147/2279] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:21:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:21:23.386878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f41c4-854e-455b-b3f6-f14b25447708 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:23.388589 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-452f41c4-854e-455b-b3f6-f14b25447708 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:21:23.455604 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f41c4-854e-455b-b3f6-f14b25447708 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:23.465438 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f83d608-62c3-4b4c-b192-e53039fef912 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08e6024b0c8420aa274653c2a916edd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:23.466157 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83d608-62c3-4b4c-b192-e53039fef912 service glance] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:23.466366 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83d608-62c3-4b4c-b192-e53039fef912 service glance] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:23.466638 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83d608-62c3-4b4c-b192-e53039fef912 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:23.477227 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f83d608-62c3-4b4c-b192-e53039fef912 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:23.478979 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f83d608-62c3-4b4c-b192-e53039fef912 service glance] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:23.489846 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1133/2280] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:21:23 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 05:21:26.776485 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34ed3e5-c4e6-4dcd-8535-4446de225e3d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:26.781202 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d34ed3e5-c4e6-4dcd-8535-4446de225e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:21:26.818227 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:21:26.888350 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34ed3e5-c4e6-4dcd-8535-4446de225e3d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:26.893480 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395537dd-5ac2-4fad-a0be-e6e663f8975d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:26.894172 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395537dd-5ac2-4fad-a0be-e6e663f8975d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:26.894452 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395537dd-5ac2-4fad-a0be-e6e663f8975d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:26.895701 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395537dd-5ac2-4fad-a0be-e6e663f8975d admin admin] PATH_INFO: `/v3/users/ae4f9098a0a8434d9bd8d495c3686594` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:26.897356 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395537dd-5ac2-4fad-a0be-e6e663f8975d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae4f9098a0a8434d9bd8d495c3686594)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:26.904389 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395537dd-5ac2-4fad-a0be-e6e663f8975d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:26.969653 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-395537dd-5ac2-4fad-a0be-e6e663f8975d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae4f9098a0a8434d9bd8d495c3686594'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:26.978359 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-395537dd-5ac2-4fad-a0be-e6e663f8975d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae4f9098a0a8434d9bd8d495c3686594'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:26.988094 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-395537dd-5ac2-4fad-a0be-e6e663f8975d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae4f9098a0a8434d9bd8d495c3686594'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:26.988760 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-395537dd-5ac2-4fad-a0be-e6e663f8975d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae4f9098a0a8434d9bd8d495c3686594'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:27.002890 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-395537dd-5ac2-4fad-a0be-e6e663f8975d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae4f9098a0a8434d9bd8d495c3686594'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:27.012701 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1148/2281] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:21:26 2022] DELETE /identity/v3/users/ae4f9098a0a8434d9bd8d495c3686594 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:21:27.213169 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f83d608-62c3-4b4c-b192-e53039fef912 service glance] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:27.232376 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f83d608-62c3-4b4c-b192-e53039fef912 service glance] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:27.276239 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:21:27.323211 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a75ec76-4acf-43af-8413-ec8f939e72a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:27.323954 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75ec76-4acf-43af-8413-ec8f939e72a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:27.324229 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75ec76-4acf-43af-8413-ec8f939e72a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:27.324496 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75ec76-4acf-43af-8413-ec8f939e72a4 admin admin] PATH_INFO: `/v3/projects/6c40b2e1ebd54a4198b60ef3e3ec1d91` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:27.325605 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a75ec76-4acf-43af-8413-ec8f939e72a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c40b2e1ebd54a4198b60ef3e3ec1d91)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:27.327546 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a75ec76-4acf-43af-8413-ec8f939e72a4 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:27.381444 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0a75ec76-4acf-43af-8413-ec8f939e72a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c40b2e1ebd54a4198b60ef3e3ec1d91', 'request_id': 'req-0a75ec76-4acf-43af-8413-ec8f939e72a4'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:27.388326 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0a75ec76-4acf-43af-8413-ec8f939e72a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c40b2e1ebd54a4198b60ef3e3ec1d91', 'request_id': 'req-0a75ec76-4acf-43af-8413-ec8f939e72a4'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:27.394181 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0a75ec76-4acf-43af-8413-ec8f939e72a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c40b2e1ebd54a4198b60ef3e3ec1d91', 'request_id': 'req-0a75ec76-4acf-43af-8413-ec8f939e72a4'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:27.408022 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0a75ec76-4acf-43af-8413-ec8f939e72a4 admin admin] The token cache is being invalidate because project 6c40b2e1ebd54a4198b60ef3e3ec1d91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:21:27.408312 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0a75ec76-4acf-43af-8413-ec8f939e72a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:27.408966 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0a75ec76-4acf-43af-8413-ec8f939e72a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:27.410776 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1134/2282] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:21:27 2022] DELETE /identity/v3/projects/6c40b2e1ebd54a4198b60ef3e3ec1d91 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:21:27.414808 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395537dd-5ac2-4fad-a0be-e6e663f8975d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:27.416322 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-395537dd-5ac2-4fad-a0be-e6e663f8975d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:21:27.429722 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395537dd-5ac2-4fad-a0be-e6e663f8975d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:27.493288 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8756c466-d2ab-469a-8d24-71fa420eabe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:27.494219 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756c466-d2ab-469a-8d24-71fa420eabe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:27.494550 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756c466-d2ab-469a-8d24-71fa420eabe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:27.494809 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756c466-d2ab-469a-8d24-71fa420eabe2 admin admin] PATH_INFO: `/v3/users/40968db1a35c4e55b002a949ede63cd9` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:27.496013 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8756c466-d2ab-469a-8d24-71fa420eabe2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40968db1a35c4e55b002a949ede63cd9)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:27.497932 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8756c466-d2ab-469a-8d24-71fa420eabe2 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:27.549065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:21:27.550707 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8756c466-d2ab-469a-8d24-71fa420eabe2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40968db1a35c4e55b002a949ede63cd9'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:27.559243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8756c466-d2ab-469a-8d24-71fa420eabe2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40968db1a35c4e55b002a949ede63cd9'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:27.568642 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8756c466-d2ab-469a-8d24-71fa420eabe2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40968db1a35c4e55b002a949ede63cd9'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:27.569528 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8756c466-d2ab-469a-8d24-71fa420eabe2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40968db1a35c4e55b002a949ede63cd9'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:27.582192 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8756c466-d2ab-469a-8d24-71fa420eabe2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40968db1a35c4e55b002a949ede63cd9'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:27.591304 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1149/2283] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:21:27 2022] DELETE /identity/v3/users/40968db1a35c4e55b002a949ede63cd9 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:21:27.775449 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a75ec76-4acf-43af-8413-ec8f939e72a4 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:27.776938 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-0a75ec76-4acf-43af-8413-ec8f939e72a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:21:27.795418 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a75ec76-4acf-43af-8413-ec8f939e72a4 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:27.858646 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d5a6e-c53b-4bea-bf63-7ced031e392f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:27.859324 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d5a6e-c53b-4bea-bf63-7ced031e392f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:27.859595 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d5a6e-c53b-4bea-bf63-7ced031e392f admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:27.859859 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d5a6e-c53b-4bea-bf63-7ced031e392f admin admin] PATH_INFO: `/v3/projects/033bafebbcd04c5a889ded54a1767cdd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:27.860987 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88d5a6e-c53b-4bea-bf63-7ced031e392f admin admin] RBAC: Authorizing `identity:delete_project(project_id=033bafebbcd04c5a889ded54a1767cdd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:27.862956 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88d5a6e-c53b-4bea-bf63-7ced031e392f admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:27.914442 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a88d5a6e-c53b-4bea-bf63-7ced031e392f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '033bafebbcd04c5a889ded54a1767cdd', 'request_id': 'req-a88d5a6e-c53b-4bea-bf63-7ced031e392f'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:27.915605 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:21:27.922753 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a88d5a6e-c53b-4bea-bf63-7ced031e392f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '033bafebbcd04c5a889ded54a1767cdd', 'request_id': 'req-a88d5a6e-c53b-4bea-bf63-7ced031e392f'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:27.928972 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a88d5a6e-c53b-4bea-bf63-7ced031e392f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '033bafebbcd04c5a889ded54a1767cdd', 'request_id': 'req-a88d5a6e-c53b-4bea-bf63-7ced031e392f'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:27.942951 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a88d5a6e-c53b-4bea-bf63-7ced031e392f admin admin] The token cache is being invalidate because project 033bafebbcd04c5a889ded54a1767cdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:21:27.943225 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a88d5a6e-c53b-4bea-bf63-7ced031e392f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:27.943846 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a88d5a6e-c53b-4bea-bf63-7ced031e392f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:27.945369 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1135/2284] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:21:27 2022] DELETE /identity/v3/projects/033bafebbcd04c5a889ded54a1767cdd => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:21:27.951420 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b816e3-3a39-465d-a0a0-620b947ef3cf None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:27.951966 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b816e3-3a39-465d-a0a0-620b947ef3cf None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:27.952421 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b816e3-3a39-465d-a0a0-620b947ef3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:27.973326 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-39b816e3-3a39-465d-a0a0-620b947ef3cf None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:21:27.974582 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-39b816e3-3a39-465d-a0a0-620b947ef3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:21:28.045088 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1150/2285] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:21:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:21:28.055030 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c849f-db7a-4b3e-b32b-cd7c85e03017 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:28.055640 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c849f-db7a-4b3e-b32b-cd7c85e03017 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:28.056138 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c849f-db7a-4b3e-b32b-cd7c85e03017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:28.083089 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-879c849f-db7a-4b3e-b32b-cd7c85e03017 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:21:28.084509 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-879c849f-db7a-4b3e-b32b-cd7c85e03017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:21:28.093412 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1136/2286] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:21:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:21:28.097998 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b816e3-3a39-465d-a0a0-620b947ef3cf None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:28.109335 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b816e3-3a39-465d-a0a0-620b947ef3cf None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:28.114619 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45b13ca-a116-46ee-a6ef-f511b262f100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:28.115311 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b13ca-a116-46ee-a6ef-f511b262f100 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:28.115565 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b13ca-a116-46ee-a6ef-f511b262f100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:28.115846 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b13ca-a116-46ee-a6ef-f511b262f100 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:28.116527 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45b13ca-a116-46ee-a6ef-f511b262f100 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:21:28.116831 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45b13ca-a116-46ee-a6ef-f511b262f100 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:28.118841 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45b13ca-a116-46ee-a6ef-f511b262f100 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:28.140831 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1151/2287] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:21:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:21:28.142531 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c849f-db7a-4b3e-b32b-cd7c85e03017 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:28.157733 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c849f-db7a-4b3e-b32b-cd7c85e03017 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:28.163991 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a0a451-d99c-4045-a145-7d38334ab689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:28.164722 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0a451-d99c-4045-a145-7d38334ab689 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:28.165095 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0a451-d99c-4045-a145-7d38334ab689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:28.165503 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0a451-d99c-4045-a145-7d38334ab689 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:28.166277 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a0a451-d99c-4045-a145-7d38334ab689 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:28.168690 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a0a451-d99c-4045-a145-7d38334ab689 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:28.209946 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1137/2288] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:21:28 2022] POST /identity/v3/projects => generated 375 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:21:28.216270 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45b13ca-a116-46ee-a6ef-f511b262f100 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:28.227224 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45b13ca-a116-46ee-a6ef-f511b262f100 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:28.257384 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f59a2-23b9-4e2a-992c-324aa56893f6 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:28.257860 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f59a2-23b9-4e2a-992c-324aa56893f6 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:28.258538 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f59a2-23b9-4e2a-992c-324aa56893f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:28.288633 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-6b5f59a2-23b9-4e2a-992c-324aa56893f6 None None] MFA Rules not processed for user `9d7f48e1763e4ff3b77ddb90e65045b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:21:28.289964 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-6b5f59a2-23b9-4e2a-992c-324aa56893f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:21:28.328820 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5fbf4e-7537-4e6c-9cef-4efec2c321d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:28.329920 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5fbf4e-7537-4e6c-9cef-4efec2c321d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:28.329920 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5fbf4e-7537-4e6c-9cef-4efec2c321d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:28.330149 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5fbf4e-7537-4e6c-9cef-4efec2c321d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:28.330625 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5fbf4e-7537-4e6c-9cef-4efec2c321d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:28.333960 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5fbf4e-7537-4e6c-9cef-4efec2c321d6 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:28.372025 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1152/2289] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:21:28 2022] POST /identity/v3/users => generated 288 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:21:28.379625 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5fbf4e-7537-4e6c-9cef-4efec2c321d6 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:28.390053 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1138/2290] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:21:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:21:28.401529 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5fbf4e-7537-4e6c-9cef-4efec2c321d6 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:28.407950 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffcbf80-347a-4db4-b6ec-f06d50ac370b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:28.409871 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffcbf80-347a-4db4-b6ec-f06d50ac370b admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:28.410298 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffcbf80-347a-4db4-b6ec-f06d50ac370b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:28.410760 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffcbf80-347a-4db4-b6ec-f06d50ac370b admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:28.411727 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffcbf80-347a-4db4-b6ec-f06d50ac370b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:28.414870 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffcbf80-347a-4db4-b6ec-f06d50ac370b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:28.437520 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1153/2291] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:21:28.441432 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5f59a2-23b9-4e2a-992c-324aa56893f6 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:28.456021 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5f59a2-23b9-4e2a-992c-324aa56893f6 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:28.460722 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978e6dfc-96f7-4ef8-9c3f-3bf09fc244c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:28.461341 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e6dfc-96f7-4ef8-9c3f-3bf09fc244c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:28.461764 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e6dfc-96f7-4ef8-9c3f-3bf09fc244c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:28.462040 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e6dfc-96f7-4ef8-9c3f-3bf09fc244c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:28.462750 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978e6dfc-96f7-4ef8-9c3f-3bf09fc244c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:28.464354 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978e6dfc-96f7-4ef8-9c3f-3bf09fc244c7 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:28.478684 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1139/2292] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:21:28.482333 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffcbf80-347a-4db4-b6ec-f06d50ac370b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:28.493725 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffcbf80-347a-4db4-b6ec-f06d50ac370b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:28.499418 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60223733-a75d-48f2-8eda-68348de09776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:28.500783 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60223733-a75d-48f2-8eda-68348de09776 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:28.501105 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60223733-a75d-48f2-8eda-68348de09776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:28.501382 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60223733-a75d-48f2-8eda-68348de09776 admin admin] PATH_INFO: `/v3/projects/a4f948b4ee2943e38807ae3e661f91b7/users/70c264fdd1184f728c05e2fce5a25e2e/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:28.517934 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60223733-a75d-48f2-8eda-68348de09776 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4f948b4ee2943e38807ae3e661f91b7, user_id=70c264fdd1184f728c05e2fce5a25e2e, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:28.520650 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60223733-a75d-48f2-8eda-68348de09776 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:28.541989 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1154/2293] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:21:28 2022] PUT /identity/v3/projects/a4f948b4ee2943e38807ae3e661f91b7/users/70c264fdd1184f728c05e2fce5a25e2e/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:21:28.553591 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f29e6-66fa-43be-8ce9-9d60ec6889ea None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:28.553875 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f29e6-66fa-43be-8ce9-9d60ec6889ea None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:28.554180 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f29e6-66fa-43be-8ce9-9d60ec6889ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:28.598040 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-0d0f29e6-66fa-43be-8ce9-9d60ec6889ea None None] MFA Rules not processed for user `70c264fdd1184f728c05e2fce5a25e2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:21:28.600151 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-0d0f29e6-66fa-43be-8ce9-9d60ec6889ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:21:28.685258 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1140/2294] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:21:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:21:28.696046 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60223733-a75d-48f2-8eda-68348de09776 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:28.697762 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-60223733-a75d-48f2-8eda-68348de09776 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:21:28.704364 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:21:28.767989 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60223733-a75d-48f2-8eda-68348de09776 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:28.772059 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526121f5-51a0-4903-bc6c-34221d9b4bb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:28.772673 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526121f5-51a0-4903-bc6c-34221d9b4bb8 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:28.772948 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526121f5-51a0-4903-bc6c-34221d9b4bb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:28.773213 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526121f5-51a0-4903-bc6c-34221d9b4bb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:28.786093 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526121f5-51a0-4903-bc6c-34221d9b4bb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:28.787552 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526121f5-51a0-4903-bc6c-34221d9b4bb8 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:28.797748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1155/2295] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:21:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:21:38.682338 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=80,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:21:38.793101 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=24 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:21:40.398946 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f29e6-66fa-43be-8ce9-9d60ec6889ea None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:40.465247 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f29e6-66fa-43be-8ce9-9d60ec6889ea None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:40.470109 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb1016-061a-4e0e-bcb1-ac0979cd6c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:40.470739 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb1016-061a-4e0e-bcb1-ac0979cd6c4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:40.471021 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb1016-061a-4e0e-bcb1-ac0979cd6c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:40.471291 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb1016-061a-4e0e-bcb1-ac0979cd6c4c admin admin] PATH_INFO: `/v3/users/70c264fdd1184f728c05e2fce5a25e2e` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:40.472438 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceb1016-061a-4e0e-bcb1-ac0979cd6c4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=70c264fdd1184f728c05e2fce5a25e2e)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:40.474406 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceb1016-061a-4e0e-bcb1-ac0979cd6c4c admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:40.528535 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-aceb1016-061a-4e0e-bcb1-ac0979cd6c4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70c264fdd1184f728c05e2fce5a25e2e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:40.535625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-aceb1016-061a-4e0e-bcb1-ac0979cd6c4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70c264fdd1184f728c05e2fce5a25e2e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:40.536358 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-aceb1016-061a-4e0e-bcb1-ac0979cd6c4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70c264fdd1184f728c05e2fce5a25e2e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:40.545254 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-aceb1016-061a-4e0e-bcb1-ac0979cd6c4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70c264fdd1184f728c05e2fce5a25e2e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:40.545909 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-aceb1016-061a-4e0e-bcb1-ac0979cd6c4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70c264fdd1184f728c05e2fce5a25e2e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:40.563316 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1141/2296] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:21:40 2022] DELETE /identity/v3/users/70c264fdd1184f728c05e2fce5a25e2e => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:21:40.568506 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313edfe-2e42-4aa4-ac3f-479bd3ccd1e7 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:40.569018 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313edfe-2e42-4aa4-ac3f-479bd3ccd1e7 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:40.569018 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313edfe-2e42-4aa4-ac3f-479bd3ccd1e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:40.591193 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-8313edfe-2e42-4aa4-ac3f-479bd3ccd1e7 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:21:40.592512 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8313edfe-2e42-4aa4-ac3f-479bd3ccd1e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:21:40.657987 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1156/2297] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:21:40.667995 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb1016-061a-4e0e-bcb1-ac0979cd6c4c admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:40.669452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-aceb1016-061a-4e0e-bcb1-ac0979cd6c4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:21:40.730665 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb1016-061a-4e0e-bcb1-ac0979cd6c4c admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:40.735058 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82872564-3ba5-48bf-ad8f-298b51f8666b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:40.735670 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82872564-3ba5-48bf-ad8f-298b51f8666b service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:40.735943 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82872564-3ba5-48bf-ad8f-298b51f8666b service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:40.736209 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82872564-3ba5-48bf-ad8f-298b51f8666b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:40.746445 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82872564-3ba5-48bf-ad8f-298b51f8666b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:40.747889 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82872564-3ba5-48bf-ad8f-298b51f8666b service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:40.757857 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1142/2298] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:21:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:21:41.030296 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8313edfe-2e42-4aa4-ac3f-479bd3ccd1e7 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:41.031910 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8313edfe-2e42-4aa4-ac3f-479bd3ccd1e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:21:41.046282 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8313edfe-2e42-4aa4-ac3f-479bd3ccd1e7 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:41.052072 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa452f67-6576-4bde-a0b9-2655bdb3ee64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:41.052736 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa452f67-6576-4bde-a0b9-2655bdb3ee64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:41.053013 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa452f67-6576-4bde-a0b9-2655bdb3ee64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:41.053283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa452f67-6576-4bde-a0b9-2655bdb3ee64 admin admin] PATH_INFO: `/v3/projects/a4f948b4ee2943e38807ae3e661f91b7` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:41.054456 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa452f67-6576-4bde-a0b9-2655bdb3ee64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4f948b4ee2943e38807ae3e661f91b7)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:41.056463 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa452f67-6576-4bde-a0b9-2655bdb3ee64 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:41.106929 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-aa452f67-6576-4bde-a0b9-2655bdb3ee64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4f948b4ee2943e38807ae3e661f91b7', 'request_id': 'req-aa452f67-6576-4bde-a0b9-2655bdb3ee64'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:41.113569 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-aa452f67-6576-4bde-a0b9-2655bdb3ee64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4f948b4ee2943e38807ae3e661f91b7', 'request_id': 'req-aa452f67-6576-4bde-a0b9-2655bdb3ee64'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:41.123611 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-aa452f67-6576-4bde-a0b9-2655bdb3ee64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4f948b4ee2943e38807ae3e661f91b7', 'request_id': 'req-aa452f67-6576-4bde-a0b9-2655bdb3ee64'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:41.135411 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-aa452f67-6576-4bde-a0b9-2655bdb3ee64 admin admin] The token cache is being invalidate because project a4f948b4ee2943e38807ae3e661f91b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:21:41.135688 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-aa452f67-6576-4bde-a0b9-2655bdb3ee64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:41.136329 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-aa452f67-6576-4bde-a0b9-2655bdb3ee64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:21:41.137861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1157/2299] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:21:41 2022] DELETE /identity/v3/projects/a4f948b4ee2943e38807ae3e661f91b7 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:21:41.143085 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5f00d-fb06-4c96-8987-618cd56b785f None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:41.143436 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5f00d-fb06-4c96-8987-618cd56b785f None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:41.143767 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5f00d-fb06-4c96-8987-618cd56b785f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:41.164215 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-c4c5f00d-fb06-4c96-8987-618cd56b785f None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:21:41.165530 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c4c5f00d-fb06-4c96-8987-618cd56b785f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:21:41.192074 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:21:41.258202 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1143/2300] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:21:41.260904 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b1ded-c38d-459b-801d-0a738c309cfb None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:41.261227 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b1ded-c38d-459b-801d-0a738c309cfb None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:41.261582 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b1ded-c38d-459b-801d-0a738c309cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:41.286496 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ac4b1ded-c38d-459b-801d-0a738c309cfb None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:21:41.287728 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ac4b1ded-c38d-459b-801d-0a738c309cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:21:41.299166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1158/2301] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:21:41.303836 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c5f00d-fb06-4c96-8987-618cd56b785f None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:41.313756 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c5f00d-fb06-4c96-8987-618cd56b785f None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:41.318434 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a23e4b3-8c66-4324-bc4e-95700985a936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:41.319038 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a23e4b3-8c66-4324-bc4e-95700985a936 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:41.319313 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a23e4b3-8c66-4324-bc4e-95700985a936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:41.319579 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a23e4b3-8c66-4324-bc4e-95700985a936 admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:41.320232 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a23e4b3-8c66-4324-bc4e-95700985a936 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:21:41.320528 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a23e4b3-8c66-4324-bc4e-95700985a936 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:41.322184 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a23e4b3-8c66-4324-bc4e-95700985a936 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:41.334630 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1144/2302] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:21:41.339237 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b1ded-c38d-459b-801d-0a738c309cfb None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:41.348591 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b1ded-c38d-459b-801d-0a738c309cfb None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:41.353255 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c723bf-0cab-4999-9adc-0c8e42beb325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:41.353893 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c723bf-0cab-4999-9adc-0c8e42beb325 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:41.354199 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c723bf-0cab-4999-9adc-0c8e42beb325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:41.354472 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c723bf-0cab-4999-9adc-0c8e42beb325 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:41.355106 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c723bf-0cab-4999-9adc-0c8e42beb325 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:41.357072 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c723bf-0cab-4999-9adc-0c8e42beb325 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:41.390051 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1159/2303] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:21:41 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:21:41.391377 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a23e4b3-8c66-4324-bc4e-95700985a936 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:41.402315 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a23e4b3-8c66-4324-bc4e-95700985a936 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:41.468491 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b5cc6c-2a9e-4bf6-a078-d5c5deb87fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:41.469286 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5cc6c-2a9e-4bf6-a078-d5c5deb87fc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:41.469655 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5cc6c-2a9e-4bf6-a078-d5c5deb87fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:41.470044 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5cc6c-2a9e-4bf6-a078-d5c5deb87fc1 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:41.470834 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b5cc6c-2a9e-4bf6-a078-d5c5deb87fc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:41.472887 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b5cc6c-2a9e-4bf6-a078-d5c5deb87fc1 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:41.497213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1145/2304] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:21:41 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:21:41.501496 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c723bf-0cab-4999-9adc-0c8e42beb325 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:41.512813 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c723bf-0cab-4999-9adc-0c8e42beb325 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:41.518289 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afebeab9-7368-4274-b045-a239d7ecb24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:41.518966 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afebeab9-7368-4274-b045-a239d7ecb24c admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:41.519316 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afebeab9-7368-4274-b045-a239d7ecb24c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:41.519654 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afebeab9-7368-4274-b045-a239d7ecb24c admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:41.520375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afebeab9-7368-4274-b045-a239d7ecb24c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:41.522065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afebeab9-7368-4274-b045-a239d7ecb24c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:41.536739 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1160/2305] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:21:41.538611 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b5cc6c-2a9e-4bf6-a078-d5c5deb87fc1 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:41.551135 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b5cc6c-2a9e-4bf6-a078-d5c5deb87fc1 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:41.570094 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba843302-ef4b-4790-bbe4-5fe3727e5278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:41.571094 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba843302-ef4b-4790-bbe4-5fe3727e5278 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:41.571641 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba843302-ef4b-4790-bbe4-5fe3727e5278 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:41.572129 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba843302-ef4b-4790-bbe4-5fe3727e5278 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:41.573377 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba843302-ef4b-4790-bbe4-5fe3727e5278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:41.576888 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba843302-ef4b-4790-bbe4-5fe3727e5278 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:41.590387 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1146/2306] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:21:41.594883 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afebeab9-7368-4274-b045-a239d7ecb24c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:41.606361 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afebeab9-7368-4274-b045-a239d7ecb24c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:41.614761 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921ec27d-07a5-4c43-8c27-ee34eb807ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:41.615447 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ec27d-07a5-4c43-8c27-ee34eb807ff7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:41.615809 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ec27d-07a5-4c43-8c27-ee34eb807ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:41.616085 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ec27d-07a5-4c43-8c27-ee34eb807ff7 admin admin] PATH_INFO: `/v3/projects/4ad69415b75d4d259c8cd31e05b2a505/users/57600fcb074741e782f0eda3ae51e6dd/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:41.637758 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921ec27d-07a5-4c43-8c27-ee34eb807ff7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ad69415b75d4d259c8cd31e05b2a505, user_id=57600fcb074741e782f0eda3ae51e6dd, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:41.641618 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921ec27d-07a5-4c43-8c27-ee34eb807ff7 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:41.663237 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1161/2307] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:21:41 2022] PUT /identity/v3/projects/4ad69415b75d4d259c8cd31e05b2a505/users/57600fcb074741e782f0eda3ae51e6dd/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:21:41.669101 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600930e-f581-4247-bdd0-84b130e967ae None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:41.669438 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600930e-f581-4247-bdd0-84b130e967ae None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:41.669783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600930e-f581-4247-bdd0-84b130e967ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:41.690771 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-1600930e-f581-4247-bdd0-84b130e967ae None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:21:41.692116 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-1600930e-f581-4247-bdd0-84b130e967ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:21:41.727131 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921ec27d-07a5-4c43-8c27-ee34eb807ff7 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:41.733304 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-921ec27d-07a5-4c43-8c27-ee34eb807ff7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:21:41.760778 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:21:41.770486 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1147/2308] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:21:41.780909 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600930e-f581-4247-bdd0-84b130e967ae None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:41.782414 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-1600930e-f581-4247-bdd0-84b130e967ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:21:41.802692 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921ec27d-07a5-4c43-8c27-ee34eb807ff7 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:41.806601 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd7c856-2278-4cd0-8759-f58de625f931 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:41.807251 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd7c856-2278-4cd0-8759-f58de625f931 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:41.807532 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd7c856-2278-4cd0-8759-f58de625f931 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:41.807807 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd7c856-2278-4cd0-8759-f58de625f931 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:41.809863 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-dbd7c856-2278-4cd0-8759-f58de625f931 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:21:41.813979 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:21:41.842750 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600930e-f581-4247-bdd0-84b130e967ae None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:41.847156 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8060cd5-662e-42e1-a290-3fca5708771a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:41.847765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8060cd5-662e-42e1-a290-3fca5708771a service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:41.848034 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8060cd5-662e-42e1-a290-3fca5708771a service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:41.848302 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8060cd5-662e-42e1-a290-3fca5708771a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:41.858456 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8060cd5-662e-42e1-a290-3fca5708771a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:41.859912 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8060cd5-662e-42e1-a290-3fca5708771a service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:41.869468 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1148/2309] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:21:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:21:41.873451 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd7c856-2278-4cd0-8759-f58de625f931 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:41.875087 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd7c856-2278-4cd0-8759-f58de625f931 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:41.885138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1162/2310] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:21:41 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:21:46.984708 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419b7c2-8b84-465d-abe9-2e40e189c5f5 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:46.985274 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419b7c2-8b84-465d-abe9-2e40e189c5f5 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:46.985380 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419b7c2-8b84-465d-abe9-2e40e189c5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:47.041269 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-a419b7c2-8b84-465d-abe9-2e40e189c5f5 None None] MFA Rules not processed for user `57600fcb074741e782f0eda3ae51e6dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:21:47.042879 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a419b7c2-8b84-465d-abe9-2e40e189c5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:21:47.125303 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1149/2311] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:21:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:21:47.130076 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd7c856-2278-4cd0-8759-f58de625f931 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:47.141829 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd7c856-2278-4cd0-8759-f58de625f931 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:47.148307 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ab7985-2a10-4b76-b63c-80e7916c34c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:47.148985 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab7985-2a10-4b76-b63c-80e7916c34c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:47.149307 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab7985-2a10-4b76-b63c-80e7916c34c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:47.149579 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab7985-2a10-4b76-b63c-80e7916c34c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:47.150322 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ab7985-2a10-4b76-b63c-80e7916c34c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:47.152569 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ab7985-2a10-4b76-b63c-80e7916c34c1 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:47.191152 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1163/2312] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:21:47 2022] POST /identity/v3/projects => generated 385 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:21:47.195042 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a419b7c2-8b84-465d-abe9-2e40e189c5f5 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:47.215233 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a419b7c2-8b84-465d-abe9-2e40e189c5f5 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:47.284320 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d830377-a1b8-443f-8d69-88db63833161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:47.284912 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d830377-a1b8-443f-8d69-88db63833161 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:47.285184 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d830377-a1b8-443f-8d69-88db63833161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:47.285457 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d830377-a1b8-443f-8d69-88db63833161 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:47.286374 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d830377-a1b8-443f-8d69-88db63833161 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:47.288086 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d830377-a1b8-443f-8d69-88db63833161 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:47.315000 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1150/2313] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:21:47 2022] POST /identity/v3/users => generated 299 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:21:47.319100 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ab7985-2a10-4b76-b63c-80e7916c34c1 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:47.329756 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ab7985-2a10-4b76-b63c-80e7916c34c1 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:47.334894 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad3b5fb-70da-485c-a2c3-322b3888c3b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:47.335543 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad3b5fb-70da-485c-a2c3-322b3888c3b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:47.335800 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad3b5fb-70da-485c-a2c3-322b3888c3b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:47.336064 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad3b5fb-70da-485c-a2c3-322b3888c3b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:47.336717 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad3b5fb-70da-485c-a2c3-322b3888c3b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:47.338482 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad3b5fb-70da-485c-a2c3-322b3888c3b4 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:47.350932 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1164/2314] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:21:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:21:47.356090 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d830377-a1b8-443f-8d69-88db63833161 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:47.367208 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d830377-a1b8-443f-8d69-88db63833161 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:47.372087 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82527bb-5358-42f8-a6d9-50274459baab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:47.372734 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82527bb-5358-42f8-a6d9-50274459baab admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:47.373509 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82527bb-5358-42f8-a6d9-50274459baab admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:47.373509 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82527bb-5358-42f8-a6d9-50274459baab admin admin] PATH_INFO: `/v3/projects/554993f9094d481d98a5cdff1b1387fa/users/627e8ed7cbfe48fe8ed21bb0333ea28c/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:47.388489 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82527bb-5358-42f8-a6d9-50274459baab admin admin] RBAC: Authorizing `identity:create_grant(project_id=554993f9094d481d98a5cdff1b1387fa, user_id=627e8ed7cbfe48fe8ed21bb0333ea28c, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:47.391127 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82527bb-5358-42f8-a6d9-50274459baab admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:47.411468 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1151/2315] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:21:47 2022] PUT /identity/v3/projects/554993f9094d481d98a5cdff1b1387fa/users/627e8ed7cbfe48fe8ed21bb0333ea28c/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:21:47.415656 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad3b5fb-70da-485c-a2c3-322b3888c3b4 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:47.425831 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad3b5fb-70da-485c-a2c3-322b3888c3b4 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:47.483217 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef3206-679e-4964-b0c4-d4f3f3ad705e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:47.483828 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef3206-679e-4964-b0c4-d4f3f3ad705e admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:47.484102 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef3206-679e-4964-b0c4-d4f3f3ad705e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:47.484363 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef3206-679e-4964-b0c4-d4f3f3ad705e admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:47.484991 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecef3206-679e-4964-b0c4-d4f3f3ad705e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:47.487915 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecef3206-679e-4964-b0c4-d4f3f3ad705e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:47.500755 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1165/2316] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:21:47 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:21:47.502666 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82527bb-5358-42f8-a6d9-50274459baab admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:47.515809 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82527bb-5358-42f8-a6d9-50274459baab admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:47.521655 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe79a0d3-5fe1-4f70-9639-ec503b8a78ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:47.522605 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe79a0d3-5fe1-4f70-9639-ec503b8a78ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:47.523007 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe79a0d3-5fe1-4f70-9639-ec503b8a78ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:47.523384 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe79a0d3-5fe1-4f70-9639-ec503b8a78ed admin admin] PATH_INFO: `/v3/projects/554993f9094d481d98a5cdff1b1387fa/users/627e8ed7cbfe48fe8ed21bb0333ea28c/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:47.525579 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe79a0d3-5fe1-4f70-9639-ec503b8a78ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=554993f9094d481d98a5cdff1b1387fa, user_id=627e8ed7cbfe48fe8ed21bb0333ea28c, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:47.528585 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe79a0d3-5fe1-4f70-9639-ec503b8a78ed admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:47.539675 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1152/2317] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:21:47 2022] PUT /identity/v3/projects/554993f9094d481d98a5cdff1b1387fa/users/627e8ed7cbfe48fe8ed21bb0333ea28c/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:21:52.730393 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1f7cd-983d-4fcb-a614-8489c8f62978 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:52.730935 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1f7cd-983d-4fcb-a614-8489c8f62978 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:52.731271 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1f7cd-983d-4fcb-a614-8489c8f62978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:52.768456 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-3fa1f7cd-983d-4fcb-a614-8489c8f62978 None None] MFA Rules not processed for user `627e8ed7cbfe48fe8ed21bb0333ea28c`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:21:52.770059 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3fa1f7cd-983d-4fcb-a614-8489c8f62978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:21:52.793257 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:21:52.858625 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1166/2318] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:21:52 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:21:52.872533 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe79a0d3-5fe1-4f70-9639-ec503b8a78ed admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:52.874433 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-fe79a0d3-5fe1-4f70-9639-ec503b8a78ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:21:52.890839 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:21:52.937166 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe79a0d3-5fe1-4f70-9639-ec503b8a78ed admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:52.941093 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b6ad79-c45e-4d9a-9e5d-71af1d117f00 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08e6024b0c8420aa274653c2a916edd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:52.941739 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6ad79-c45e-4d9a-9e5d-71af1d117f00 service glance] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:52.942029 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6ad79-c45e-4d9a-9e5d-71af1d117f00 service glance] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:52.942313 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6ad79-c45e-4d9a-9e5d-71af1d117f00 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:53.006956 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b6ad79-c45e-4d9a-9e5d-71af1d117f00 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:53.008449 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b6ad79-c45e-4d9a-9e5d-71af1d117f00 service glance] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:53.018032 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1153/2319] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:21:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:21:53.182526 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955f240-7af4-44f5-bef3-30025cf4a182 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:53.182812 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955f240-7af4-44f5-bef3-30025cf4a182 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:53.183076 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955f240-7af4-44f5-bef3-30025cf4a182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:53.205467 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-0955f240-7af4-44f5-bef3-30025cf4a182 None None] MFA Rules not processed for user `09bf6db88ce8494a937211a814c9ff84`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:21:53.206984 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0955f240-7af4-44f5-bef3-30025cf4a182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:21:53.282476 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1167/2320] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Aug 31 05:21:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:21:53.329137 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b6ad79-c45e-4d9a-9e5d-71af1d117f00 service glance] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:53.330774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-e1b6ad79-c45e-4d9a-9e5d-71af1d117f00 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:21:53.416176 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b6ad79-c45e-4d9a-9e5d-71af1d117f00 service glance] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:53.421670 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be040fad-1387-42a9-86e3-92fe6e5246e3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9b82080a73d4c4a8ec08b0e67d6d11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:53.422321 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be040fad-1387-42a9-86e3-92fe6e5246e3 service swift] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:53.422603 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be040fad-1387-42a9-86e3-92fe6e5246e3 service swift] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:53.422874 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be040fad-1387-42a9-86e3-92fe6e5246e3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:53.432186 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be040fad-1387-42a9-86e3-92fe6e5246e3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:53.433959 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be040fad-1387-42a9-86e3-92fe6e5246e3 service swift] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:53.445242 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1154/2321] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Aug 31 05:21:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 05:21:53.731673 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f94227-6f49-49b3-9a1c-8c14c3fa6682 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:53.732931 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f94227-6f49-49b3-9a1c-8c14c3fa6682 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:53.733340 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f94227-6f49-49b3-9a1c-8c14c3fa6682 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:53.735391 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1168/2322] 10.222.0.36 () {54 vars in 961 bytes} [Wed Aug 31 05:21:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:21:53.741510 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a6e1f-5f08-4c46-8035-606958653c20 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:53.742130 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a6e1f-5f08-4c46-8035-606958653c20 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:53.742562 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a6e1f-5f08-4c46-8035-606958653c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:53.773897 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-d70a6e1f-5f08-4c46-8035-606958653c20 None None] MFA Rules not processed for user `bd4387035099416789009f0ecb8ad282`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:21:53.783198 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-d70a6e1f-5f08-4c46-8035-606958653c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:21:53.891186 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:21:53.939517 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1155/2323] 10.222.0.36 () {58 vars in 1058 bytes} [Wed Aug 31 05:21:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:21:53.947658 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f94227-6f49-49b3-9a1c-8c14c3fa6682 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:21:53.970440 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f94227-6f49-49b3-9a1c-8c14c3fa6682 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:21:53.978971 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37deaf49-3839-4a6c-9497-372e1d8a583a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd4387035099416789009f0ecb8ad282', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:21:53.979650 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37deaf49-3839-4a6c-9497-372e1d8a583a service placement] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:21:53.979935 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37deaf49-3839-4a6c-9497-372e1d8a583a service placement] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:21:53.980203 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37deaf49-3839-4a6c-9497-372e1d8a583a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:21:53.984983 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-37deaf49-3839-4a6c-9497-372e1d8a583a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:21:54.000860 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37deaf49-3839-4a6c-9497-372e1d8a583a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:21:54.002890 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37deaf49-3839-4a6c-9497-372e1d8a583a service placement] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:21:54.014542 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1169/2324] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:21:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:22:03.906961 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:22:03.950821 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba41ee-2ccb-4519-b68f-024512e2fdf4 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:03.951151 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba41ee-2ccb-4519-b68f-024512e2fdf4 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:03.951422 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba41ee-2ccb-4519-b68f-024512e2fdf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:03.973240 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-c9ba41ee-2ccb-4519-b68f-024512e2fdf4 None None] MFA Rules not processed for user `57600fcb074741e782f0eda3ae51e6dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:22:03.974501 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c9ba41ee-2ccb-4519-b68f-024512e2fdf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:22:03.983060 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1156/2325] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:22:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:22:04.009501 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=50 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:22:04.760837 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c557a-e3b7-4293-a3ef-a0434f22b9f3 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:04.761215 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c557a-e3b7-4293-a3ef-a0434f22b9f3 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:04.763363 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c557a-e3b7-4293-a3ef-a0434f22b9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:04.785919 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-681c557a-e3b7-4293-a3ef-a0434f22b9f3 None None] MFA Rules not processed for user `09bf6db88ce8494a937211a814c9ff84`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:22:04.787415 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-681c557a-e3b7-4293-a3ef-a0434f22b9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:22:04.798351 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1170/2326] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Aug 31 05:22:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:22:04.808933 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ba41ee-2ccb-4519-b68f-024512e2fdf4 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:22:04.827524 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ba41ee-2ccb-4519-b68f-024512e2fdf4 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:22:04.832739 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3623ea-a570-4b03-b5a4-7824dd33d893 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9b82080a73d4c4a8ec08b0e67d6d11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:22:04.833618 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3623ea-a570-4b03-b5a4-7824dd33d893 service swift] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:04.834127 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3623ea-a570-4b03-b5a4-7824dd33d893 service swift] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:04.834402 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3623ea-a570-4b03-b5a4-7824dd33d893 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:04.843460 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3623ea-a570-4b03-b5a4-7824dd33d893 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:22:04.845522 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3623ea-a570-4b03-b5a4-7824dd33d893 service swift] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:22:04.854577 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1157/2327] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Aug 31 05:22:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 05:22:12.615436 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d7b8c1-bc7c-4862-b330-c53ff77820b0 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:12.616028 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d7b8c1-bc7c-4862-b330-c53ff77820b0 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:12.616028 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d7b8c1-bc7c-4862-b330-c53ff77820b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:12.646589 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-98d7b8c1-bc7c-4862-b330-c53ff77820b0 None None] MFA Rules not processed for user `09bf6db88ce8494a937211a814c9ff84`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:22:12.648107 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-98d7b8c1-bc7c-4862-b330-c53ff77820b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:22:12.658880 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1171/2328] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Aug 31 05:22:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:22:12.676982 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3623ea-a570-4b03-b5a4-7824dd33d893 service swift] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:22:12.691912 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3623ea-a570-4b03-b5a4-7824dd33d893 service swift] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:22:12.705904 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c65cc1-cba5-42cc-bc2d-ee1fcead5759 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9b82080a73d4c4a8ec08b0e67d6d11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:22:12.706846 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c65cc1-cba5-42cc-bc2d-ee1fcead5759 service swift] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:12.707346 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c65cc1-cba5-42cc-bc2d-ee1fcead5759 service swift] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:12.707884 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c65cc1-cba5-42cc-bc2d-ee1fcead5759 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:12.718211 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c65cc1-cba5-42cc-bc2d-ee1fcead5759 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:22:12.720490 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c65cc1-cba5-42cc-bc2d-ee1fcead5759 service swift] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:22:12.730618 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1158/2329] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Aug 31 05:22:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 05:22:18.611140 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d7b8c1-bc7c-4862-b330-c53ff77820b0 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:22:18.612742 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-98d7b8c1-bc7c-4862-b330-c53ff77820b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:22:18.762524 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d7b8c1-bc7c-4862-b330-c53ff77820b0 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:22:18.768794 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e40f3c-c674-4d99-bbd0-48b6de0fb3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:22:18.769488 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e40f3c-c674-4d99-bbd0-48b6de0fb3c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:18.769776 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e40f3c-c674-4d99-bbd0-48b6de0fb3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:18.770083 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e40f3c-c674-4d99-bbd0-48b6de0fb3c6 admin admin] PATH_INFO: `/v3/users/20cf2265f6dd47678ee70ecb2d84a15a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:18.771275 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e40f3c-c674-4d99-bbd0-48b6de0fb3c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20cf2265f6dd47678ee70ecb2d84a15a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:22:18.773499 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e40f3c-c674-4d99-bbd0-48b6de0fb3c6 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:22:18.855913 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-58e40f3c-c674-4d99-bbd0-48b6de0fb3c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20cf2265f6dd47678ee70ecb2d84a15a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:22:18.865846 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-58e40f3c-c674-4d99-bbd0-48b6de0fb3c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20cf2265f6dd47678ee70ecb2d84a15a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:22:18.879376 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-58e40f3c-c674-4d99-bbd0-48b6de0fb3c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20cf2265f6dd47678ee70ecb2d84a15a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:22:18.893497 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-58e40f3c-c674-4d99-bbd0-48b6de0fb3c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20cf2265f6dd47678ee70ecb2d84a15a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:22:18.937143 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-58e40f3c-c674-4d99-bbd0-48b6de0fb3c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20cf2265f6dd47678ee70ecb2d84a15a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:22:18.965449 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1d1e0-34e1-4545-9e32-ac920fad8f40 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:18.966516 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1d1e0-34e1-4545-9e32-ac920fad8f40 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:18.967243 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1d1e0-34e1-4545-9e32-ac920fad8f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:18.970233 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1172/2330] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:22:18 2022] DELETE /identity/v3/users/20cf2265f6dd47678ee70ecb2d84a15a => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:22:19.014037 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-23b1d1e0-34e1-4545-9e32-ac920fad8f40 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:22:19.015482 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-23b1d1e0-34e1-4545-9e32-ac920fad8f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:22:19.179379 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1159/2331] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:22:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:22:19.192325 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e40f3c-c674-4d99-bbd0-48b6de0fb3c6 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:22:19.193960 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-58e40f3c-c674-4d99-bbd0-48b6de0fb3c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:22:19.335191 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e40f3c-c674-4d99-bbd0-48b6de0fb3c6 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:22:19.343138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d7c072-acdd-48dc-a337-cf814b87e54f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:22:19.344463 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7c072-acdd-48dc-a337-cf814b87e54f service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:19.345115 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7c072-acdd-48dc-a337-cf814b87e54f service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:19.345762 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7c072-acdd-48dc-a337-cf814b87e54f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:19.360730 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d7c072-acdd-48dc-a337-cf814b87e54f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:22:19.362596 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d7c072-acdd-48dc-a337-cf814b87e54f service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:22:19.380006 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1173/2332] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:22:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:22:19.835857 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b1d1e0-34e1-4545-9e32-ac920fad8f40 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:22:19.837818 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-23b1d1e0-34e1-4545-9e32-ac920fad8f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:22:19.862338 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b1d1e0-34e1-4545-9e32-ac920fad8f40 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:22:19.877735 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db05e1-db6d-4d70-9fc9-b9936eb6c2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:22:19.878418 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db05e1-db6d-4d70-9fc9-b9936eb6c2c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:19.878692 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db05e1-db6d-4d70-9fc9-b9936eb6c2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:19.878959 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db05e1-db6d-4d70-9fc9-b9936eb6c2c5 admin admin] PATH_INFO: `/v3/projects/fca7e32ef9214c44a34f72790208a668` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:19.881350 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33db05e1-db6d-4d70-9fc9-b9936eb6c2c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fca7e32ef9214c44a34f72790208a668)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:22:19.888831 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33db05e1-db6d-4d70-9fc9-b9936eb6c2c5 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:22:19.987244 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-33db05e1-db6d-4d70-9fc9-b9936eb6c2c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fca7e32ef9214c44a34f72790208a668', 'request_id': 'req-33db05e1-db6d-4d70-9fc9-b9936eb6c2c5'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:22:20.005085 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-33db05e1-db6d-4d70-9fc9-b9936eb6c2c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fca7e32ef9214c44a34f72790208a668', 'request_id': 'req-33db05e1-db6d-4d70-9fc9-b9936eb6c2c5'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:22:20.022408 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-33db05e1-db6d-4d70-9fc9-b9936eb6c2c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fca7e32ef9214c44a34f72790208a668', 'request_id': 'req-33db05e1-db6d-4d70-9fc9-b9936eb6c2c5'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:22:20.053797 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-33db05e1-db6d-4d70-9fc9-b9936eb6c2c5 admin admin] The token cache is being invalidate because project fca7e32ef9214c44a34f72790208a668 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:22:20.056933 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-33db05e1-db6d-4d70-9fc9-b9936eb6c2c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:22:20.058730 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-33db05e1-db6d-4d70-9fc9-b9936eb6c2c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:22:20.065870 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1160/2333] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:22:19 2022] DELETE /identity/v3/projects/fca7e32ef9214c44a34f72790208a668 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:22:20.068543 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3c73f-27a8-47c1-b15a-15d8bf5b3030 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:20.069060 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3c73f-27a8-47c1-b15a-15d8bf5b3030 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:20.069396 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3c73f-27a8-47c1-b15a-15d8bf5b3030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:20.121743 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a7a3c73f-27a8-47c1-b15a-15d8bf5b3030 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:22:20.123322 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a7a3c73f-27a8-47c1-b15a-15d8bf5b3030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:22:20.144761 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:22:20.260398 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1174/2334] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:22:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:22:20.267830 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce9c900-65fa-4e15-bbf7-20da5a17b71a None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:20.268110 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce9c900-65fa-4e15-bbf7-20da5a17b71a None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:20.268395 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce9c900-65fa-4e15-bbf7-20da5a17b71a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:20.304765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-7ce9c900-65fa-4e15-bbf7-20da5a17b71a None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:22:20.306381 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-7ce9c900-65fa-4e15-bbf7-20da5a17b71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:22:20.319212 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1161/2335] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:22:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:22:20.319786 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a3c73f-27a8-47c1-b15a-15d8bf5b3030 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:22:20.348545 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a3c73f-27a8-47c1-b15a-15d8bf5b3030 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:22:20.367258 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c6163-e9ba-4d7f-8b4b-e4c6691f874e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:22:20.368047 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c6163-e9ba-4d7f-8b4b-e4c6691f874e admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:20.368577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c6163-e9ba-4d7f-8b4b-e4c6691f874e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:20.368907 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c6163-e9ba-4d7f-8b4b-e4c6691f874e admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:20.369689 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36c6163-e9ba-4d7f-8b4b-e4c6691f874e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:22:20.370194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36c6163-e9ba-4d7f-8b4b-e4c6691f874e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:22:20.375173 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36c6163-e9ba-4d7f-8b4b-e4c6691f874e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:22:20.409178 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1175/2336] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:22:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:22:20.413443 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce9c900-65fa-4e15-bbf7-20da5a17b71a None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:22:20.428876 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce9c900-65fa-4e15-bbf7-20da5a17b71a None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:22:20.435484 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339fbf11-0d17-4c12-b12b-ec06edef6b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:22:20.436597 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339fbf11-0d17-4c12-b12b-ec06edef6b91 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:20.436887 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339fbf11-0d17-4c12-b12b-ec06edef6b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:20.437152 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339fbf11-0d17-4c12-b12b-ec06edef6b91 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:20.437815 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339fbf11-0d17-4c12-b12b-ec06edef6b91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:22:20.439913 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339fbf11-0d17-4c12-b12b-ec06edef6b91 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:22:20.511590 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c6163-e9ba-4d7f-8b4b-e4c6691f874e admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:22:20.525226 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1162/2337] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:22:20 2022] POST /identity/v3/projects => generated 371 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:22:20.534166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c6163-e9ba-4d7f-8b4b-e4c6691f874e admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:22:20.699706 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0080f0a7-a89c-42cd-a2f6-c11a1327ef0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:22:20.700335 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080f0a7-a89c-42cd-a2f6-c11a1327ef0a admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:20.700604 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080f0a7-a89c-42cd-a2f6-c11a1327ef0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:20.700872 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080f0a7-a89c-42cd-a2f6-c11a1327ef0a admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:20.701522 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0080f0a7-a89c-42cd-a2f6-c11a1327ef0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:22:20.703487 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0080f0a7-a89c-42cd-a2f6-c11a1327ef0a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:22:20.730918 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1176/2338] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:22:20 2022] POST /identity/v3/users => generated 286 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:22:20.738321 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339fbf11-0d17-4c12-b12b-ec06edef6b91 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:22:20.750433 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339fbf11-0d17-4c12-b12b-ec06edef6b91 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:22:20.756911 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218bb565-2257-417a-b1ea-0820c4f98d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:22:20.757880 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218bb565-2257-417a-b1ea-0820c4f98d25 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:20.758464 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218bb565-2257-417a-b1ea-0820c4f98d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:20.758962 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218bb565-2257-417a-b1ea-0820c4f98d25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:20.759764 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218bb565-2257-417a-b1ea-0820c4f98d25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:22:20.767043 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218bb565-2257-417a-b1ea-0820c4f98d25 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:22:20.781308 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1163/2339] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:22:20.785000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0080f0a7-a89c-42cd-a2f6-c11a1327ef0a admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:22:20.796005 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0080f0a7-a89c-42cd-a2f6-c11a1327ef0a admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:22:20.802691 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568953a-4ac8-4cb5-bb3c-ca8ff327469a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:22:20.803545 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568953a-4ac8-4cb5-bb3c-ca8ff327469a admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:20.804119 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568953a-4ac8-4cb5-bb3c-ca8ff327469a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:20.804569 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568953a-4ac8-4cb5-bb3c-ca8ff327469a admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:20.805757 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b568953a-4ac8-4cb5-bb3c-ca8ff327469a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:22:20.808935 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b568953a-4ac8-4cb5-bb3c-ca8ff327469a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:22:20.826044 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1177/2340] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:22:20.829569 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218bb565-2257-417a-b1ea-0820c4f98d25 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:22:20.840527 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218bb565-2257-417a-b1ea-0820c4f98d25 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:22:20.849822 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91be11c-b403-4a40-8872-cdc6822bdb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:22:20.854852 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91be11c-b403-4a40-8872-cdc6822bdb58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:20.855188 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91be11c-b403-4a40-8872-cdc6822bdb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:20.855404 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91be11c-b403-4a40-8872-cdc6822bdb58 admin admin] PATH_INFO: `/v3/projects/1bb219690bf5464c9f88a8647f41340d/users/7f33a93833b04e64bf269d2c126698f0/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:20.865795 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:22:20.927556 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91be11c-b403-4a40-8872-cdc6822bdb58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bb219690bf5464c9f88a8647f41340d, user_id=7f33a93833b04e64bf269d2c126698f0, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:22:20.930635 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91be11c-b403-4a40-8872-cdc6822bdb58 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:22:20.976848 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1164/2341] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:22:20 2022] PUT /identity/v3/projects/1bb219690bf5464c9f88a8647f41340d/users/7f33a93833b04e64bf269d2c126698f0/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:22:20.990879 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91edaa-9208-4b16-bed9-2eb05f6cecd4 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:20.991193 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91edaa-9208-4b16-bed9-2eb05f6cecd4 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:20.991518 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91edaa-9208-4b16-bed9-2eb05f6cecd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:21.021835 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-2e91edaa-9208-4b16-bed9-2eb05f6cecd4 None None] MFA Rules not processed for user `7f33a93833b04e64bf269d2c126698f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:22:21.023367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2e91edaa-9208-4b16-bed9-2eb05f6cecd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:22:21.138138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1178/2342] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:22:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:22:21.893409 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91be11c-b403-4a40-8872-cdc6822bdb58 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:22:21.895360 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a91be11c-b403-4a40-8872-cdc6822bdb58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:22:21.960131 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91be11c-b403-4a40-8872-cdc6822bdb58 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:22:21.965544 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d4d1f-2af1-4ea4-b653-15f6a4d8476b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08e6024b0c8420aa274653c2a916edd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:22:21.966197 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d4d1f-2af1-4ea4-b653-15f6a4d8476b service glance] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:21.966477 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d4d1f-2af1-4ea4-b653-15f6a4d8476b service glance] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:21.966749 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d4d1f-2af1-4ea4-b653-15f6a4d8476b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:21.975636 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d4d1f-2af1-4ea4-b653-15f6a4d8476b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:22:21.977173 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d4d1f-2af1-4ea4-b653-15f6a4d8476b service glance] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:22:21.987863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1165/2343] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:22:21 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:22:27.337031 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2b058-6a40-45cb-ae3e-fb7a7c243100 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:27.337609 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2b058-6a40-45cb-ae3e-fb7a7c243100 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:27.337715 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2b058-6a40-45cb-ae3e-fb7a7c243100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:27.349945 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:22:27.362418 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b7d2b058-6a40-45cb-ae3e-fb7a7c243100 None None] MFA Rules not processed for user `09bf6db88ce8494a937211a814c9ff84`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:22:27.363649 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b7d2b058-6a40-45cb-ae3e-fb7a7c243100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:22:27.426115 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1179/2344] 10.222.0.36 () {58 vars in 967 bytes} [Wed Aug 31 05:22:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:22:27.439205 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d4d1f-2af1-4ea4-b653-15f6a4d8476b service glance] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:22:27.440835 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-fa6d4d1f-2af1-4ea4-b653-15f6a4d8476b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:22:27.517494 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d4d1f-2af1-4ea4-b653-15f6a4d8476b service glance] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:22:27.523262 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b93c63-e11d-409e-8536-dfde8456221a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9b82080a73d4c4a8ec08b0e67d6d11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:22:27.523908 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b93c63-e11d-409e-8536-dfde8456221a service swift] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:27.524182 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b93c63-e11d-409e-8536-dfde8456221a service swift] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:27.524474 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b93c63-e11d-409e-8536-dfde8456221a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:27.534522 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b93c63-e11d-409e-8536-dfde8456221a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:22:27.536204 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b93c63-e11d-409e-8536-dfde8456221a service swift] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:22:27.545294 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1166/2345] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Aug 31 05:22:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 05:22:29.594576 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2b058-6a40-45cb-ae3e-fb7a7c243100 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:22:29.596958 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b7d2b058-6a40-45cb-ae3e-fb7a7c243100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:22:29.682063 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2b058-6a40-45cb-ae3e-fb7a7c243100 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:22:29.687325 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed06acd-fcce-44fd-8ae5-87e4471cdfd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:22:29.688010 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed06acd-fcce-44fd-8ae5-87e4471cdfd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:29.688290 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed06acd-fcce-44fd-8ae5-87e4471cdfd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:29.688569 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed06acd-fcce-44fd-8ae5-87e4471cdfd7 admin admin] PATH_INFO: `/v3/users/9d7f48e1763e4ff3b77ddb90e65045b0` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:29.690110 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed06acd-fcce-44fd-8ae5-87e4471cdfd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d7f48e1763e4ff3b77ddb90e65045b0)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:22:29.692111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed06acd-fcce-44fd-8ae5-87e4471cdfd7 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:22:29.750261 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7ed06acd-fcce-44fd-8ae5-87e4471cdfd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d7f48e1763e4ff3b77ddb90e65045b0'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:22:29.758204 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7ed06acd-fcce-44fd-8ae5-87e4471cdfd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d7f48e1763e4ff3b77ddb90e65045b0'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:22:29.767797 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7ed06acd-fcce-44fd-8ae5-87e4471cdfd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d7f48e1763e4ff3b77ddb90e65045b0'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:22:29.768563 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7ed06acd-fcce-44fd-8ae5-87e4471cdfd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d7f48e1763e4ff3b77ddb90e65045b0'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:22:29.781696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7ed06acd-fcce-44fd-8ae5-87e4471cdfd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d7f48e1763e4ff3b77ddb90e65045b0'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:22:29.787991 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1180/2346] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:22:29 2022] DELETE /identity/v3/users/9d7f48e1763e4ff3b77ddb90e65045b0 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:22:29.975889 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b93c63-e11d-409e-8536-dfde8456221a service swift] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:22:29.991931 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b93c63-e11d-409e-8536-dfde8456221a service swift] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:22:30.073743 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff0fda-882d-4e41-a5fb-c4f9d99146f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:22:30.074437 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff0fda-882d-4e41-a5fb-c4f9d99146f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:30.074721 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff0fda-882d-4e41-a5fb-c4f9d99146f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:30.074991 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff0fda-882d-4e41-a5fb-c4f9d99146f6 admin admin] PATH_INFO: `/v3/projects/04da0465aa794d208d6dd9e2aab7d704` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:30.076464 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff0fda-882d-4e41-a5fb-c4f9d99146f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04da0465aa794d208d6dd9e2aab7d704)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:22:30.078635 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff0fda-882d-4e41-a5fb-c4f9d99146f6 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:22:30.105427 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:22:30.146352 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-05ff0fda-882d-4e41-a5fb-c4f9d99146f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04da0465aa794d208d6dd9e2aab7d704', 'request_id': 'req-05ff0fda-882d-4e41-a5fb-c4f9d99146f6'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:22:30.152941 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-05ff0fda-882d-4e41-a5fb-c4f9d99146f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04da0465aa794d208d6dd9e2aab7d704', 'request_id': 'req-05ff0fda-882d-4e41-a5fb-c4f9d99146f6'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:22:30.158965 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-05ff0fda-882d-4e41-a5fb-c4f9d99146f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04da0465aa794d208d6dd9e2aab7d704', 'request_id': 'req-05ff0fda-882d-4e41-a5fb-c4f9d99146f6'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:22:30.174297 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-05ff0fda-882d-4e41-a5fb-c4f9d99146f6 admin admin] The token cache is being invalidate because project 04da0465aa794d208d6dd9e2aab7d704 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:22:30.174583 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-05ff0fda-882d-4e41-a5fb-c4f9d99146f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:22:30.175219 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-05ff0fda-882d-4e41-a5fb-c4f9d99146f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:22:30.180109 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1167/2347] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:22:29 2022] DELETE /identity/v3/projects/04da0465aa794d208d6dd9e2aab7d704 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:22:30.181764 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af29751-f625-4bc2-a779-c2fae1e68c24 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:30.182074 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af29751-f625-4bc2-a779-c2fae1e68c24 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:30.182345 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af29751-f625-4bc2-a779-c2fae1e68c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:30.204593 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-7af29751-f625-4bc2-a779-c2fae1e68c24 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:22:30.205896 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7af29751-f625-4bc2-a779-c2fae1e68c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:22:30.229550 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:22:30.277637 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1181/2348] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:22:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:22:30.285594 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea12bf-a62c-4700-8c41-8130ed8a3860 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:30.285945 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea12bf-a62c-4700-8c41-8130ed8a3860 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:30.286399 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea12bf-a62c-4700-8c41-8130ed8a3860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:30.306854 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-81ea12bf-a62c-4700-8c41-8130ed8a3860 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:22:30.308060 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-81ea12bf-a62c-4700-8c41-8130ed8a3860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:22:30.317193 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1168/2349] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:22:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:22:30.321192 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af29751-f625-4bc2-a779-c2fae1e68c24 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:22:30.333051 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af29751-f625-4bc2-a779-c2fae1e68c24 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:22:30.338312 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d78310-7627-440a-bef0-80daf9d61af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:22:30.338938 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d78310-7627-440a-bef0-80daf9d61af4 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:30.339212 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d78310-7627-440a-bef0-80daf9d61af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:30.339477 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d78310-7627-440a-bef0-80daf9d61af4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:30.340127 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d78310-7627-440a-bef0-80daf9d61af4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:22:30.340426 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d78310-7627-440a-bef0-80daf9d61af4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:22:30.342172 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d78310-7627-440a-bef0-80daf9d61af4 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:22:30.358835 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1182/2350] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:22:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:22:30.360016 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ea12bf-a62c-4700-8c41-8130ed8a3860 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:22:30.370040 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ea12bf-a62c-4700-8c41-8130ed8a3860 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:22:30.375914 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1edf7c3-ee43-44db-a1a8-07f943f30bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:22:30.376546 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1edf7c3-ee43-44db-a1a8-07f943f30bb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:30.376819 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1edf7c3-ee43-44db-a1a8-07f943f30bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:30.377257 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1edf7c3-ee43-44db-a1a8-07f943f30bb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:30.377907 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1edf7c3-ee43-44db-a1a8-07f943f30bb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:22:30.379911 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1edf7c3-ee43-44db-a1a8-07f943f30bb7 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:22:30.410607 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1169/2351] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:22:30 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:22:30.415275 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d78310-7627-440a-bef0-80daf9d61af4 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:22:30.426084 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d78310-7627-440a-bef0-80daf9d61af4 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:22:30.502769 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f41d79-04ed-4567-bd48-d3ef8a68f3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:22:30.503764 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f41d79-04ed-4567-bd48-d3ef8a68f3a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:30.503764 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f41d79-04ed-4567-bd48-d3ef8a68f3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:30.504080 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f41d79-04ed-4567-bd48-d3ef8a68f3a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:30.504592 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f41d79-04ed-4567-bd48-d3ef8a68f3a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:22:30.507086 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f41d79-04ed-4567-bd48-d3ef8a68f3a9 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:22:30.530380 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1183/2352] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:22:30 2022] POST /identity/v3/users => generated 286 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:22:30.534346 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1edf7c3-ee43-44db-a1a8-07f943f30bb7 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:22:30.548147 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1edf7c3-ee43-44db-a1a8-07f943f30bb7 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:22:30.553364 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c619d9-d436-4b51-bb6f-e5249d7c9810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:22:30.553989 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c619d9-d436-4b51-bb6f-e5249d7c9810 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:30.554290 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c619d9-d436-4b51-bb6f-e5249d7c9810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:30.554555 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c619d9-d436-4b51-bb6f-e5249d7c9810 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:30.555404 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c619d9-d436-4b51-bb6f-e5249d7c9810 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:22:30.557018 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c619d9-d436-4b51-bb6f-e5249d7c9810 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:22:30.568254 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1170/2353] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:22:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 05:22:30.572447 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f41d79-04ed-4567-bd48-d3ef8a68f3a9 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:22:30.583099 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f41d79-04ed-4567-bd48-d3ef8a68f3a9 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:22:30.587989 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13405ff-0181-411c-bab5-ef454b045424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:22:30.588603 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13405ff-0181-411c-bab5-ef454b045424 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:30.588871 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13405ff-0181-411c-bab5-ef454b045424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:30.589152 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13405ff-0181-411c-bab5-ef454b045424 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:30.589798 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13405ff-0181-411c-bab5-ef454b045424 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:22:30.591525 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13405ff-0181-411c-bab5-ef454b045424 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:22:30.612123 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1184/2354] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:22:30 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:22:30.612812 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c619d9-d436-4b51-bb6f-e5249d7c9810 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:22:30.622983 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c619d9-d436-4b51-bb6f-e5249d7c9810 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:22:30.628013 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4818dd05-eb63-48a3-9cb3-e6fb604ee98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:22:30.628956 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4818dd05-eb63-48a3-9cb3-e6fb604ee98d admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:30.628956 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4818dd05-eb63-48a3-9cb3-e6fb604ee98d admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:30.629196 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4818dd05-eb63-48a3-9cb3-e6fb604ee98d admin admin] PATH_INFO: `/v3/projects/400a8bd1863140f58b0b70fcd902d628/users/7fcf72b1bb7c4bc18930d6b99cd7e3eb/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:30.648901 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4818dd05-eb63-48a3-9cb3-e6fb604ee98d admin admin] RBAC: Authorizing `identity:create_grant(project_id=400a8bd1863140f58b0b70fcd902d628, user_id=7fcf72b1bb7c4bc18930d6b99cd7e3eb, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:22:30.651875 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4818dd05-eb63-48a3-9cb3-e6fb604ee98d admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:22:30.668199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1171/2355] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:22:30 2022] PUT /identity/v3/projects/400a8bd1863140f58b0b70fcd902d628/users/7fcf72b1bb7c4bc18930d6b99cd7e3eb/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:22:30.683777 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cc793-67f8-42fd-a3b7-5949c5fbb726 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:30.684055 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cc793-67f8-42fd-a3b7-5949c5fbb726 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:30.684322 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cc793-67f8-42fd-a3b7-5949c5fbb726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:30.713191 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ed1cc793-67f8-42fd-a3b7-5949c5fbb726 None None] MFA Rules not processed for user `7fcf72b1bb7c4bc18930d6b99cd7e3eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:22:30.714451 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ed1cc793-67f8-42fd-a3b7-5949c5fbb726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:22:30.782389 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1185/2356] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:22:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:22:30.793509 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4818dd05-eb63-48a3-9cb3-e6fb604ee98d admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:22:30.795361 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-4818dd05-eb63-48a3-9cb3-e6fb604ee98d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:22:30.860784 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4818dd05-eb63-48a3-9cb3-e6fb604ee98d admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:22:30.864629 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85852c4-6ef6-43f0-b082-77f061649022 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:22:30.865240 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85852c4-6ef6-43f0-b082-77f061649022 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:30.865536 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85852c4-6ef6-43f0-b082-77f061649022 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:30.865806 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85852c4-6ef6-43f0-b082-77f061649022 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:30.879583 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85852c4-6ef6-43f0-b082-77f061649022 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:22:30.881108 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85852c4-6ef6-43f0-b082-77f061649022 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:22:30.891359 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1172/2357] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:22:30 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:22:30.920410 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1cc793-67f8-42fd-a3b7-5949c5fbb726 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:22:30.954302 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:22:31.000634 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1cc793-67f8-42fd-a3b7-5949c5fbb726 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:22:31.005876 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405aace0-90f5-4c9b-8e3a-9039dd0b4ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:22:31.006744 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405aace0-90f5-4c9b-8e3a-9039dd0b4ddf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:31.007108 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405aace0-90f5-4c9b-8e3a-9039dd0b4ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:31.007373 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405aace0-90f5-4c9b-8e3a-9039dd0b4ddf admin admin] PATH_INFO: `/v3/users/7fcf72b1bb7c4bc18930d6b99cd7e3eb` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:31.008589 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405aace0-90f5-4c9b-8e3a-9039dd0b4ddf admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fcf72b1bb7c4bc18930d6b99cd7e3eb)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:22:31.010560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405aace0-90f5-4c9b-8e3a-9039dd0b4ddf admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:22:31.066343 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-405aace0-90f5-4c9b-8e3a-9039dd0b4ddf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fcf72b1bb7c4bc18930d6b99cd7e3eb'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:22:31.074031 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-405aace0-90f5-4c9b-8e3a-9039dd0b4ddf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fcf72b1bb7c4bc18930d6b99cd7e3eb'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:22:31.083074 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-405aace0-90f5-4c9b-8e3a-9039dd0b4ddf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fcf72b1bb7c4bc18930d6b99cd7e3eb'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:22:31.083674 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-405aace0-90f5-4c9b-8e3a-9039dd0b4ddf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fcf72b1bb7c4bc18930d6b99cd7e3eb'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:22:31.095703 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-405aace0-90f5-4c9b-8e3a-9039dd0b4ddf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fcf72b1bb7c4bc18930d6b99cd7e3eb'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:22:31.102373 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1186/2358] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:22:30 2022] DELETE /identity/v3/users/7fcf72b1bb7c4bc18930d6b99cd7e3eb => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:22:31.107451 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a20bdf3-8975-48cd-bbda-bcada2c0d32f None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:31.107938 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a20bdf3-8975-48cd-bbda-bcada2c0d32f None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:31.108032 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a20bdf3-8975-48cd-bbda-bcada2c0d32f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:31.126956 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-7a20bdf3-8975-48cd-bbda-bcada2c0d32f None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:22:31.128135 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-7a20bdf3-8975-48cd-bbda-bcada2c0d32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:22:31.178730 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:22:31.196844 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1173/2359] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:22:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:22:31.206972 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405aace0-90f5-4c9b-8e3a-9039dd0b4ddf admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:22:31.208410 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-405aace0-90f5-4c9b-8e3a-9039dd0b4ddf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:22:31.276234 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405aace0-90f5-4c9b-8e3a-9039dd0b4ddf admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:22:31.279967 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3b442f-f31e-416d-a660-0283827fa144 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:22:31.280604 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b442f-f31e-416d-a660-0283827fa144 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:31.280885 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b442f-f31e-416d-a660-0283827fa144 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:31.281149 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b442f-f31e-416d-a660-0283827fa144 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:31.291107 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3b442f-f31e-416d-a660-0283827fa144 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:22:31.292514 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3b442f-f31e-416d-a660-0283827fa144 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:22:31.303423 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1187/2360] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:22:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:22:31.599174 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a20bdf3-8975-48cd-bbda-bcada2c0d32f None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:22:31.600812 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-7a20bdf3-8975-48cd-bbda-bcada2c0d32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:22:31.615100 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a20bdf3-8975-48cd-bbda-bcada2c0d32f None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:22:31.620799 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a89038e-4fbc-4fab-ab3f-a313e5db9f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:22:31.621470 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a89038e-4fbc-4fab-ab3f-a313e5db9f53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:31.621743 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a89038e-4fbc-4fab-ab3f-a313e5db9f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:31.622194 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a89038e-4fbc-4fab-ab3f-a313e5db9f53 admin admin] PATH_INFO: `/v3/projects/400a8bd1863140f58b0b70fcd902d628` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:31.624379 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a89038e-4fbc-4fab-ab3f-a313e5db9f53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=400a8bd1863140f58b0b70fcd902d628)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:22:31.636622 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a89038e-4fbc-4fab-ab3f-a313e5db9f53 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:22:31.701328 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-8a89038e-4fbc-4fab-ab3f-a313e5db9f53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '400a8bd1863140f58b0b70fcd902d628', 'request_id': 'req-8a89038e-4fbc-4fab-ab3f-a313e5db9f53'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:22:31.707767 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-8a89038e-4fbc-4fab-ab3f-a313e5db9f53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '400a8bd1863140f58b0b70fcd902d628', 'request_id': 'req-8a89038e-4fbc-4fab-ab3f-a313e5db9f53'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:22:31.713754 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-8a89038e-4fbc-4fab-ab3f-a313e5db9f53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '400a8bd1863140f58b0b70fcd902d628', 'request_id': 'req-8a89038e-4fbc-4fab-ab3f-a313e5db9f53'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:22:31.727270 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-8a89038e-4fbc-4fab-ab3f-a313e5db9f53 admin admin] The token cache is being invalidate because project 400a8bd1863140f58b0b70fcd902d628 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:22:31.727556 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-8a89038e-4fbc-4fab-ab3f-a313e5db9f53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:22:31.728249 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-8a89038e-4fbc-4fab-ab3f-a313e5db9f53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:22:31.729861 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1174/2361] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:22:31 2022] DELETE /identity/v3/projects/400a8bd1863140f58b0b70fcd902d628 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:22:31.735564 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac7bc20-c783-4278-9047-de04adf11388 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:31.736067 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac7bc20-c783-4278-9047-de04adf11388 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:31.736182 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac7bc20-c783-4278-9047-de04adf11388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:31.756764 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-fac7bc20-c783-4278-9047-de04adf11388 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:22:31.757811 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fac7bc20-c783-4278-9047-de04adf11388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:22:31.804933 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:22:31.826619 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1188/2362] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:22:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:22:31.834519 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f02da4-ed66-4662-8ecd-a0a8a13f36ba None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:31.835080 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f02da4-ed66-4662-8ecd-a0a8a13f36ba None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:31.835432 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f02da4-ed66-4662-8ecd-a0a8a13f36ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:31.856189 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-44f02da4-ed66-4662-8ecd-a0a8a13f36ba None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:22:31.857793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-44f02da4-ed66-4662-8ecd-a0a8a13f36ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:22:31.867147 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1175/2363] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:22:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:22:31.872235 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac7bc20-c783-4278-9047-de04adf11388 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:22:31.883293 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac7bc20-c783-4278-9047-de04adf11388 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:22:31.891002 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80cf0e7-8a61-43cd-9c32-a43cedb3b88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:22:31.891651 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80cf0e7-8a61-43cd-9c32-a43cedb3b88a admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:31.891927 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80cf0e7-8a61-43cd-9c32-a43cedb3b88a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:31.892191 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80cf0e7-8a61-43cd-9c32-a43cedb3b88a admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:31.893003 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80cf0e7-8a61-43cd-9c32-a43cedb3b88a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:22:31.893276 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80cf0e7-8a61-43cd-9c32-a43cedb3b88a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:22:31.894915 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80cf0e7-8a61-43cd-9c32-a43cedb3b88a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:22:31.907428 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1189/2364] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:22:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:22:31.911843 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f02da4-ed66-4662-8ecd-a0a8a13f36ba None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:22:31.921831 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f02da4-ed66-4662-8ecd-a0a8a13f36ba None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:22:31.926539 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40efe1f-f4bf-4334-bb97-fffa0580b9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:22:31.927152 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40efe1f-f4bf-4334-bb97-fffa0580b9fc admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:31.927436 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40efe1f-f4bf-4334-bb97-fffa0580b9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:31.927701 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40efe1f-f4bf-4334-bb97-fffa0580b9fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:31.928331 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40efe1f-f4bf-4334-bb97-fffa0580b9fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:22:31.930343 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40efe1f-f4bf-4334-bb97-fffa0580b9fc admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:22:31.963493 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1176/2365] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:22:31 2022] POST /identity/v3/projects => generated 397 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:22:31.964768 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80cf0e7-8a61-43cd-9c32-a43cedb3b88a admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:22:31.978268 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80cf0e7-8a61-43cd-9c32-a43cedb3b88a admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:22:32.050622 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec9538c-33e2-444a-914a-601edcc838bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:22:32.051381 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9538c-33e2-444a-914a-601edcc838bb admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:32.051656 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9538c-33e2-444a-914a-601edcc838bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:32.051925 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9538c-33e2-444a-914a-601edcc838bb admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:32.052597 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec9538c-33e2-444a-914a-601edcc838bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:22:32.054750 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec9538c-33e2-444a-914a-601edcc838bb admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:22:32.077222 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1190/2366] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:22:31 2022] POST /identity/v3/users => generated 299 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:22:32.081170 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40efe1f-f4bf-4334-bb97-fffa0580b9fc admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:22:32.091955 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40efe1f-f4bf-4334-bb97-fffa0580b9fc admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:22:32.096712 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c18fef-74cf-492e-8dd8-1481b81c5afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:22:32.097319 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c18fef-74cf-492e-8dd8-1481b81c5afd admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:32.097598 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c18fef-74cf-492e-8dd8-1481b81c5afd admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:32.097869 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c18fef-74cf-492e-8dd8-1481b81c5afd admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:32.098582 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c18fef-74cf-492e-8dd8-1481b81c5afd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:22:32.100375 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c18fef-74cf-492e-8dd8-1481b81c5afd admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:22:32.111944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1177/2367] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:22:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:22:32.116523 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec9538c-33e2-444a-914a-601edcc838bb admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:22:32.128819 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec9538c-33e2-444a-914a-601edcc838bb admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:22:32.133740 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5886a2e8-dc0d-43d6-83ef-728e4ada3d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:22:32.134394 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5886a2e8-dc0d-43d6-83ef-728e4ada3d75 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:32.134668 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5886a2e8-dc0d-43d6-83ef-728e4ada3d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:32.134933 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5886a2e8-dc0d-43d6-83ef-728e4ada3d75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:32.135599 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5886a2e8-dc0d-43d6-83ef-728e4ada3d75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:22:32.137229 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5886a2e8-dc0d-43d6-83ef-728e4ada3d75 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:22:32.148404 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1191/2368] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:22:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:22:32.152956 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c18fef-74cf-492e-8dd8-1481b81c5afd admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:22:32.163485 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c18fef-74cf-492e-8dd8-1481b81c5afd admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:22:32.168190 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526685a-fa8e-402c-9a0e-8f591b001ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:22:32.168856 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526685a-fa8e-402c-9a0e-8f591b001ddf admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:32.169128 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526685a-fa8e-402c-9a0e-8f591b001ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:32.169391 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526685a-fa8e-402c-9a0e-8f591b001ddf admin admin] PATH_INFO: `/v3/projects/439e8d9e204f4290b56214e00a88a7b6/users/86dd30dae7a24d1798afdfcbf91b2460/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:32.183751 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f526685a-fa8e-402c-9a0e-8f591b001ddf admin admin] RBAC: Authorizing `identity:create_grant(project_id=439e8d9e204f4290b56214e00a88a7b6, user_id=86dd30dae7a24d1798afdfcbf91b2460, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:22:32.186605 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f526685a-fa8e-402c-9a0e-8f591b001ddf admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:22:32.203470 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1178/2369] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:22:32 2022] PUT /identity/v3/projects/439e8d9e204f4290b56214e00a88a7b6/users/86dd30dae7a24d1798afdfcbf91b2460/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:22:32.216165 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226a3a6-3b04-4638-87e0-3683e4188ed9 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:32.216526 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226a3a6-3b04-4638-87e0-3683e4188ed9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:32.216835 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226a3a6-3b04-4638-87e0-3683e4188ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:32.251204 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e226a3a6-3b04-4638-87e0-3683e4188ed9 None None] MFA Rules not processed for user `86dd30dae7a24d1798afdfcbf91b2460`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:22:32.252924 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e226a3a6-3b04-4638-87e0-3683e4188ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:22:32.331274 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1192/2370] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:22:32 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:22:32.343202 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526685a-fa8e-402c-9a0e-8f591b001ddf admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:22:32.345053 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-f526685a-fa8e-402c-9a0e-8f591b001ddf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:22:32.404989 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526685a-fa8e-402c-9a0e-8f591b001ddf admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:22:32.408870 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ae3d3-b610-4bc8-b314-60b2165051f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:22:32.409506 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ae3d3-b610-4bc8-b314-60b2165051f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:32.409777 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ae3d3-b610-4bc8-b314-60b2165051f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:32.410053 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ae3d3-b610-4bc8-b314-60b2165051f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:32.421393 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9ae3d3-b610-4bc8-b314-60b2165051f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:22:32.422890 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9ae3d3-b610-4bc8-b314-60b2165051f8 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:22:32.434132 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1179/2371] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:22:32 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:22:42.327726 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=78,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:22:42.430456 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=85,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:22:45.898712 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e226a3a6-3b04-4638-87e0-3683e4188ed9 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:22:45.963949 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e226a3a6-3b04-4638-87e0-3683e4188ed9 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:22:45.969288 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c87cf-c0bb-461f-875d-bbbd825f5e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:22:45.969905 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c87cf-c0bb-461f-875d-bbbd825f5e61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:45.970216 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c87cf-c0bb-461f-875d-bbbd825f5e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:45.970481 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c87cf-c0bb-461f-875d-bbbd825f5e61 admin admin] PATH_INFO: `/v3/users/86dd30dae7a24d1798afdfcbf91b2460` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:45.971646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5c87cf-c0bb-461f-875d-bbbd825f5e61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86dd30dae7a24d1798afdfcbf91b2460)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:22:45.973583 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5c87cf-c0bb-461f-875d-bbbd825f5e61 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:22:46.021624 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ec5c87cf-c0bb-461f-875d-bbbd825f5e61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86dd30dae7a24d1798afdfcbf91b2460'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:22:46.028773 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ec5c87cf-c0bb-461f-875d-bbbd825f5e61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86dd30dae7a24d1798afdfcbf91b2460'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:22:46.036482 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ec5c87cf-c0bb-461f-875d-bbbd825f5e61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86dd30dae7a24d1798afdfcbf91b2460'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:22:46.037108 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ec5c87cf-c0bb-461f-875d-bbbd825f5e61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86dd30dae7a24d1798afdfcbf91b2460'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:22:46.049703 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ec5c87cf-c0bb-461f-875d-bbbd825f5e61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86dd30dae7a24d1798afdfcbf91b2460'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:22:46.058882 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1193/2372] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:22:45 2022] DELETE /identity/v3/users/86dd30dae7a24d1798afdfcbf91b2460 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:22:46.064073 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ac5b8-9245-49ee-87a9-c209ce8ac507 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:46.064597 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ac5b8-9245-49ee-87a9-c209ce8ac507 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:46.064597 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ac5b8-9245-49ee-87a9-c209ce8ac507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:46.084164 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-109ac5b8-9245-49ee-87a9-c209ce8ac507 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:22:46.085366 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-109ac5b8-9245-49ee-87a9-c209ce8ac507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:22:46.144681 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1180/2373] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:22:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:22:46.155213 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c87cf-c0bb-461f-875d-bbbd825f5e61 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:22:46.156799 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ec5c87cf-c0bb-461f-875d-bbbd825f5e61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:22:46.213745 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c87cf-c0bb-461f-875d-bbbd825f5e61 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:22:46.217878 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b80bb-56fa-4b51-8d50-0ec390dbc37e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:22:46.218524 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b80bb-56fa-4b51-8d50-0ec390dbc37e service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:46.218795 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b80bb-56fa-4b51-8d50-0ec390dbc37e service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:46.219057 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b80bb-56fa-4b51-8d50-0ec390dbc37e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:46.228395 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4b80bb-56fa-4b51-8d50-0ec390dbc37e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:22:46.229832 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4b80bb-56fa-4b51-8d50-0ec390dbc37e service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:22:46.239695 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1194/2374] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:22:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:22:46.524842 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ac5b8-9245-49ee-87a9-c209ce8ac507 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:22:46.526530 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-109ac5b8-9245-49ee-87a9-c209ce8ac507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:22:46.538906 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ac5b8-9245-49ee-87a9-c209ce8ac507 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:22:46.544222 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75887f0d-828a-43c2-8e24-9d027f6278d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:22:46.544892 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75887f0d-828a-43c2-8e24-9d027f6278d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:22:46.545257 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75887f0d-828a-43c2-8e24-9d027f6278d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:22:46.545420 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75887f0d-828a-43c2-8e24-9d027f6278d7 admin admin] PATH_INFO: `/v3/projects/439e8d9e204f4290b56214e00a88a7b6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:22:46.546643 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75887f0d-828a-43c2-8e24-9d027f6278d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=439e8d9e204f4290b56214e00a88a7b6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:22:46.548603 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75887f0d-828a-43c2-8e24-9d027f6278d7 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:22:46.608806 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-75887f0d-828a-43c2-8e24-9d027f6278d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '439e8d9e204f4290b56214e00a88a7b6', 'request_id': 'req-75887f0d-828a-43c2-8e24-9d027f6278d7'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:22:46.617600 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-75887f0d-828a-43c2-8e24-9d027f6278d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '439e8d9e204f4290b56214e00a88a7b6', 'request_id': 'req-75887f0d-828a-43c2-8e24-9d027f6278d7'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:22:46.623108 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-75887f0d-828a-43c2-8e24-9d027f6278d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '439e8d9e204f4290b56214e00a88a7b6', 'request_id': 'req-75887f0d-828a-43c2-8e24-9d027f6278d7'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:22:46.637044 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-75887f0d-828a-43c2-8e24-9d027f6278d7 admin admin] The token cache is being invalidate because project 439e8d9e204f4290b56214e00a88a7b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:22:46.637352 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-75887f0d-828a-43c2-8e24-9d027f6278d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:22:46.638163 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-75887f0d-828a-43c2-8e24-9d027f6278d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:22:46.640051 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1181/2375] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:22:46 2022] DELETE /identity/v3/projects/439e8d9e204f4290b56214e00a88a7b6 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:22:56.235449 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:22:56.634116 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:23:07.316927 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b80bb-56fa-4b51-8d50-0ec390dbc37e service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:23:07.319210 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fb4b80bb-56fa-4b51-8d50-0ec390dbc37e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:23:07.335473 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b80bb-56fa-4b51-8d50-0ec390dbc37e service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:23:07.390592 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90966055-d3b3-4f6c-8f0a-8439b8f5d3d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:23:07.391200 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90966055-d3b3-4f6c-8f0a-8439b8f5d3d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:23:07.391487 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90966055-d3b3-4f6c-8f0a-8439b8f5d3d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:23:07.391749 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90966055-d3b3-4f6c-8f0a-8439b8f5d3d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:23:07.393849 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-90966055-d3b3-4f6c-8f0a-8439b8f5d3d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:23:07.458769 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90966055-d3b3-4f6c-8f0a-8439b8f5d3d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:23:07.460147 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90966055-d3b3-4f6c-8f0a-8439b8f5d3d5 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:23:07.471067 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1195/2376] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:23:07 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:23:10.048195 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75887f0d-828a-43c2-8e24-9d027f6278d7 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:23:10.050573 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-75887f0d-828a-43c2-8e24-9d027f6278d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:23:10.125978 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75887f0d-828a-43c2-8e24-9d027f6278d7 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:23:10.131786 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66412c5-cd2b-4761-bf98-385363e49423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:23:10.132399 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66412c5-cd2b-4761-bf98-385363e49423 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:23:10.132689 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66412c5-cd2b-4761-bf98-385363e49423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:23:10.133126 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66412c5-cd2b-4761-bf98-385363e49423 admin admin] PATH_INFO: `/v3/users/7f33a93833b04e64bf269d2c126698f0` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:23:10.134448 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66412c5-cd2b-4761-bf98-385363e49423 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f33a93833b04e64bf269d2c126698f0)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:23:10.137001 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66412c5-cd2b-4761-bf98-385363e49423 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:23:10.193660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a66412c5-cd2b-4761-bf98-385363e49423 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f33a93833b04e64bf269d2c126698f0'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:23:10.202350 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a66412c5-cd2b-4761-bf98-385363e49423 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f33a93833b04e64bf269d2c126698f0'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:23:10.203117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a66412c5-cd2b-4761-bf98-385363e49423 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f33a93833b04e64bf269d2c126698f0'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:23:10.213403 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a66412c5-cd2b-4761-bf98-385363e49423 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f33a93833b04e64bf269d2c126698f0'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:23:10.214170 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a66412c5-cd2b-4761-bf98-385363e49423 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f33a93833b04e64bf269d2c126698f0'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:23:10.237985 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1182/2377] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:23:10 2022] DELETE /identity/v3/users/7f33a93833b04e64bf269d2c126698f0 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:23:10.242260 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d5d42b-433f-4bd5-91e6-373073f06668 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:23:10.242757 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d5d42b-433f-4bd5-91e6-373073f06668 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:23:10.242877 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d5d42b-433f-4bd5-91e6-373073f06668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:23:10.269425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-62d5d42b-433f-4bd5-91e6-373073f06668 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:23:10.271044 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-62d5d42b-433f-4bd5-91e6-373073f06668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:23:10.351346 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1196/2378] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:23:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:23:10.368042 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66412c5-cd2b-4761-bf98-385363e49423 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:23:10.370034 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a66412c5-cd2b-4761-bf98-385363e49423 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:23:10.445802 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66412c5-cd2b-4761-bf98-385363e49423 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:23:10.449784 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab182d4d-629d-4423-95f2-479cd3a048b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:23:10.450440 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab182d4d-629d-4423-95f2-479cd3a048b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:23:10.450713 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab182d4d-629d-4423-95f2-479cd3a048b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:23:10.451132 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab182d4d-629d-4423-95f2-479cd3a048b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:23:10.462475 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab182d4d-629d-4423-95f2-479cd3a048b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:23:10.463891 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab182d4d-629d-4423-95f2-479cd3a048b6 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:23:10.478792 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1183/2379] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:23:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:23:11.393807 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d5d42b-433f-4bd5-91e6-373073f06668 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:23:11.395901 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-62d5d42b-433f-4bd5-91e6-373073f06668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:23:11.415672 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d5d42b-433f-4bd5-91e6-373073f06668 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:23:11.420505 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ae6cb-2978-4042-9346-03f5010364d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:23:11.421141 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ae6cb-2978-4042-9346-03f5010364d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:23:11.421418 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ae6cb-2978-4042-9346-03f5010364d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:23:11.421691 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ae6cb-2978-4042-9346-03f5010364d5 admin admin] PATH_INFO: `/v3/projects/1bb219690bf5464c9f88a8647f41340d` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:23:11.422859 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4ae6cb-2978-4042-9346-03f5010364d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bb219690bf5464c9f88a8647f41340d)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:23:11.425280 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4ae6cb-2978-4042-9346-03f5010364d5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:23:11.484840 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:23:11.489229 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8e4ae6cb-2978-4042-9346-03f5010364d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bb219690bf5464c9f88a8647f41340d', 'request_id': 'req-8e4ae6cb-2978-4042-9346-03f5010364d5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:23:11.497936 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8e4ae6cb-2978-4042-9346-03f5010364d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bb219690bf5464c9f88a8647f41340d', 'request_id': 'req-8e4ae6cb-2978-4042-9346-03f5010364d5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:23:11.507038 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8e4ae6cb-2978-4042-9346-03f5010364d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bb219690bf5464c9f88a8647f41340d', 'request_id': 'req-8e4ae6cb-2978-4042-9346-03f5010364d5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:23:11.519770 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8e4ae6cb-2978-4042-9346-03f5010364d5 admin admin] The token cache is being invalidate because project 1bb219690bf5464c9f88a8647f41340d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:23:11.520059 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8e4ae6cb-2978-4042-9346-03f5010364d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:23:11.520719 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8e4ae6cb-2978-4042-9346-03f5010364d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:23:11.522471 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1197/2380] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:23:11 2022] DELETE /identity/v3/projects/1bb219690bf5464c9f88a8647f41340d => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:23:14.538858 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a281ca0f-5e82-4ee2-bbed-650934d5eff8 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:23:14.539367 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a281ca0f-5e82-4ee2-bbed-650934d5eff8 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:23:14.539594 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a281ca0f-5e82-4ee2-bbed-650934d5eff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:23:14.559739 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-a281ca0f-5e82-4ee2-bbed-650934d5eff8 None None] MFA Rules not processed for user `09bf6db88ce8494a937211a814c9ff84`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:23:14.561102 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a281ca0f-5e82-4ee2-bbed-650934d5eff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:23:14.581018 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:23:14.625117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1184/2381] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Aug 31 05:23:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:23:14.648146 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ae6cb-2978-4042-9346-03f5010364d5 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:23:14.649593 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8e4ae6cb-2978-4042-9346-03f5010364d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:23:14.720339 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ae6cb-2978-4042-9346-03f5010364d5 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:23:14.725955 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19760e57-cba5-4d29-bece-d6242f229b2e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9b82080a73d4c4a8ec08b0e67d6d11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:23:14.726682 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19760e57-cba5-4d29-bece-d6242f229b2e service swift] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:23:14.726977 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19760e57-cba5-4d29-bece-d6242f229b2e service swift] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:23:14.727272 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19760e57-cba5-4d29-bece-d6242f229b2e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:23:14.736333 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19760e57-cba5-4d29-bece-d6242f229b2e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:23:14.738184 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19760e57-cba5-4d29-bece-d6242f229b2e service swift] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:23:14.747024 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1198/2382] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Aug 31 05:23:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 31 05:23:15.404600 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa029d-5d2d-4282-9717-e59466a795fa None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:23:15.404888 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa029d-5d2d-4282-9717-e59466a795fa None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:23:15.405278 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa029d-5d2d-4282-9717-e59466a795fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:23:15.430053 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-33fa029d-5d2d-4282-9717-e59466a795fa None None] MFA Rules not processed for user `806b011517ef4201a86c90472e0740d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:23:15.431241 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-33fa029d-5d2d-4282-9717-e59466a795fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:23:15.495392 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1185/2383] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:23:15 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:23:24.744132 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:DELETE=4,keystone:SELECT=39,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:23:25.488767 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=40 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:23:29.419862 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b588cc-9daa-4865-8cb7-332466b507cb None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:23:29.420415 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b588cc-9daa-4865-8cb7-332466b507cb None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:23:29.420415 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b588cc-9daa-4865-8cb7-332466b507cb None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:23:29.421899 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1199/2384] 10.222.0.36 () {54 vars in 921 bytes} [Wed Aug 31 05:23:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:23:29.428351 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca419b02-ba97-46e2-987f-bc696a3e7410 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:23:29.428837 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca419b02-ba97-46e2-987f-bc696a3e7410 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:23:29.429072 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca419b02-ba97-46e2-987f-bc696a3e7410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:23:29.449927 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-ca419b02-ba97-46e2-987f-bc696a3e7410 None None] MFA Rules not processed for user `f27665b1574a4db7b7a9771f7d775826`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:23:29.451165 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-ca419b02-ba97-46e2-987f-bc696a3e7410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:23:29.524338 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1186/2385] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Aug 31 05:23:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:23:29.536723 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b588cc-9daa-4865-8cb7-332466b507cb None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:23:29.538540 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-09b588cc-9daa-4865-8cb7-332466b507cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:23:29.612138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b588cc-9daa-4865-8cb7-332466b507cb None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:23:29.616072 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb1bf4-1477-4b3f-a597-b2722a40ea01 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08e6024b0c8420aa274653c2a916edd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:23:29.616688 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb1bf4-1477-4b3f-a597-b2722a40ea01 service glance] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:23:29.616963 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb1bf4-1477-4b3f-a597-b2722a40ea01 service glance] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:23:29.617230 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb1bf4-1477-4b3f-a597-b2722a40ea01 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:23:29.627621 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cb1bf4-1477-4b3f-a597-b2722a40ea01 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:23:29.631110 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cb1bf4-1477-4b3f-a597-b2722a40ea01 service glance] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:23:29.648503 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1200/2386] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:23:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:23:29.717988 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c98ba3-0c40-4cb0-abf9-08e94eaaf67f None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:23:29.718325 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c98ba3-0c40-4cb0-abf9-08e94eaaf67f None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:23:29.718684 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c98ba3-0c40-4cb0-abf9-08e94eaaf67f None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:23:29.720749 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1187/2387] 10.222.0.36 () {54 vars in 921 bytes} [Wed Aug 31 05:23:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:23:29.725580 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bcae3d-e05c-49d3-b63f-d99a33624435 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:23:29.725893 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bcae3d-e05c-49d3-b63f-d99a33624435 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:23:29.726205 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bcae3d-e05c-49d3-b63f-d99a33624435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:23:29.749367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a9bcae3d-e05c-49d3-b63f-d99a33624435 None None] MFA Rules not processed for user `090e1409506542f6831e1f65d44aae74`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:23:29.750617 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a9bcae3d-e05c-49d3-b63f-d99a33624435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:23:29.811091 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1201/2388] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Aug 31 05:23:29 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:23:29.823086 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c98ba3-0c40-4cb0-abf9-08e94eaaf67f None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:23:29.825158 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c2c98ba3-0c40-4cb0-abf9-08e94eaaf67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:23:29.840135 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c98ba3-0c40-4cb0-abf9-08e94eaaf67f None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:23:29.844467 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b070d13c-de5b-4ac4-8a7a-37e93b333e5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:23:29.845064 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070d13c-de5b-4ac4-8a7a-37e93b333e5f service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:23:29.845341 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070d13c-de5b-4ac4-8a7a-37e93b333e5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:23:29.845612 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070d13c-de5b-4ac4-8a7a-37e93b333e5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:23:29.853512 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b070d13c-de5b-4ac4-8a7a-37e93b333e5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:23:29.855315 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b070d13c-de5b-4ac4-8a7a-37e93b333e5f service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:23:29.863681 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1188/2389] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:23:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:23:29.947128 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b421363-a76b-4176-a1bd-3adc081ddb11 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:23:29.947414 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b421363-a76b-4176-a1bd-3adc081ddb11 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:23:29.947683 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b421363-a76b-4176-a1bd-3adc081ddb11 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:23:29.949769 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1202/2390] 10.222.0.36 () {54 vars in 921 bytes} [Wed Aug 31 05:23:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:23:29.955192 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6332fb-f7db-4d37-8275-4ca08e535342 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:23:29.955449 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6332fb-f7db-4d37-8275-4ca08e535342 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:23:29.955852 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6332fb-f7db-4d37-8275-4ca08e535342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:23:29.976648 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-4b6332fb-f7db-4d37-8275-4ca08e535342 None None] MFA Rules not processed for user `f27665b1574a4db7b7a9771f7d775826`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:23:29.977849 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-4b6332fb-f7db-4d37-8275-4ca08e535342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:23:29.987690 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1189/2391] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Aug 31 05:23:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:23:29.998131 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b421363-a76b-4176-a1bd-3adc081ddb11 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:23:30.010735 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b421363-a76b-4176-a1bd-3adc081ddb11 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:23:30.014966 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c94f68-0a3d-4b85-bd6c-ea29e094bae5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:23:30.015561 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c94f68-0a3d-4b85-bd6c-ea29e094bae5 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:23:30.015837 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c94f68-0a3d-4b85-bd6c-ea29e094bae5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:23:30.016108 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c94f68-0a3d-4b85-bd6c-ea29e094bae5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:23:30.027337 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c94f68-0a3d-4b85-bd6c-ea29e094bae5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:23:30.028751 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c94f68-0a3d-4b85-bd6c-ea29e094bae5 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:23:30.039403 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1203/2392] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:23:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:23:34.631554 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db1bfb-a6ff-414d-8da6-0d38d0115861 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:23:34.632097 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db1bfb-a6ff-414d-8da6-0d38d0115861 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:23:34.632097 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db1bfb-a6ff-414d-8da6-0d38d0115861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:23:34.652517 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-19db1bfb-a6ff-414d-8da6-0d38d0115861 None None] MFA Rules not processed for user `09bf6db88ce8494a937211a814c9ff84`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:23:34.653727 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-19db1bfb-a6ff-414d-8da6-0d38d0115861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:23:34.661452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1190/2393] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Aug 31 05:23:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:23:34.674830 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c94f68-0a3d-4b85-bd6c-ea29e094bae5 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:23:34.687710 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c94f68-0a3d-4b85-bd6c-ea29e094bae5 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:23:34.693739 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a77cf-1cfc-4a69-a791-b6e29350159a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9b82080a73d4c4a8ec08b0e67d6d11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:23:34.694403 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a77cf-1cfc-4a69-a791-b6e29350159a service swift] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:23:34.694690 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a77cf-1cfc-4a69-a791-b6e29350159a service swift] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:23:34.695099 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a77cf-1cfc-4a69-a791-b6e29350159a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:23:34.704646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07a77cf-1cfc-4a69-a791-b6e29350159a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:23:34.706367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07a77cf-1cfc-4a69-a791-b6e29350159a service swift] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:23:34.715570 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1204/2394] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Aug 31 05:23:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 05:23:35.666212 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db1bfb-a6ff-414d-8da6-0d38d0115861 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:23:35.676072 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db1bfb-a6ff-414d-8da6-0d38d0115861 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:23:35.679794 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c071fbf-d52a-4cd7-947e-50c7af6d02ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:23:35.680394 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c071fbf-d52a-4cd7-947e-50c7af6d02ad service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:23:35.680675 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c071fbf-d52a-4cd7-947e-50c7af6d02ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:23:35.680914 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c071fbf-d52a-4cd7-947e-50c7af6d02ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:23:35.682663 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3c071fbf-d52a-4cd7-947e-50c7af6d02ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:23:35.696357 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c071fbf-d52a-4cd7-947e-50c7af6d02ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:23:35.697838 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c071fbf-d52a-4cd7-947e-50c7af6d02ad service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:23:35.709740 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1191/2395] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:23:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:23:36.891831 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a77cf-1cfc-4a69-a791-b6e29350159a service swift] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:23:36.902479 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a77cf-1cfc-4a69-a791-b6e29350159a service swift] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:23:36.907410 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5100b8a9-e144-4081-8ba6-0828d359cb04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:23:36.908017 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5100b8a9-e144-4081-8ba6-0828d359cb04 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:23:36.908292 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5100b8a9-e144-4081-8ba6-0828d359cb04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:23:36.908559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5100b8a9-e144-4081-8ba6-0828d359cb04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:23:36.910658 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5100b8a9-e144-4081-8ba6-0828d359cb04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:23:36.924282 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5100b8a9-e144-4081-8ba6-0828d359cb04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:23:36.925770 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5100b8a9-e144-4081-8ba6-0828d359cb04 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:23:36.937036 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1205/2396] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:23:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:23:45.704342 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=49 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:23:46.931971 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=67 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:23:48.808189 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c071fbf-d52a-4cd7-947e-50c7af6d02ad service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:23:48.817126 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c071fbf-d52a-4cd7-947e-50c7af6d02ad service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:23:48.821062 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08dd49a-49e9-442c-81b6-88b0cf5d9bdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:23:48.821650 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08dd49a-49e9-442c-81b6-88b0cf5d9bdd service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:23:48.821921 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08dd49a-49e9-442c-81b6-88b0cf5d9bdd service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:23:48.822222 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08dd49a-49e9-442c-81b6-88b0cf5d9bdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:23:48.824224 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c08dd49a-49e9-442c-81b6-88b0cf5d9bdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:23:48.885197 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08dd49a-49e9-442c-81b6-88b0cf5d9bdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:23:48.886684 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08dd49a-49e9-442c-81b6-88b0cf5d9bdd service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:23:48.896432 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1192/2397] 10.222.0.36 () {60 vars in 1396 bytes} [Wed Aug 31 05:23:48 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3266 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:23:58.891945 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=28 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:24:05.409062 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5100b8a9-e144-4081-8ba6-0828d359cb04 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:24:05.418363 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5100b8a9-e144-4081-8ba6-0828d359cb04 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:24:05.422759 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f392be-8fcd-484a-ac1f-d6cca33544fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:24:05.423388 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f392be-8fcd-484a-ac1f-d6cca33544fb service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:24:05.423724 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f392be-8fcd-484a-ac1f-d6cca33544fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:24:05.423985 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f392be-8fcd-484a-ac1f-d6cca33544fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:24:05.425885 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-60f392be-8fcd-484a-ac1f-d6cca33544fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:24:05.435576 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f392be-8fcd-484a-ac1f-d6cca33544fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:24:05.436988 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f392be-8fcd-484a-ac1f-d6cca33544fb service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:24:05.444925 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1206/2398] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:24:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:24:07.352075 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04a9ae-a205-479e-b7b4-45d1225b1240 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:24:07.352565 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04a9ae-a205-479e-b7b4-45d1225b1240 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:24:07.352565 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04a9ae-a205-479e-b7b4-45d1225b1240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:24:07.372975 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-3d04a9ae-a205-479e-b7b4-45d1225b1240 None None] MFA Rules not processed for user `09bf6db88ce8494a937211a814c9ff84`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:24:07.374209 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3d04a9ae-a205-479e-b7b4-45d1225b1240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:24:07.382106 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1193/2399] 10.222.0.36 () {58 vars in 967 bytes} [Wed Aug 31 05:24:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:24:07.397094 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f392be-8fcd-484a-ac1f-d6cca33544fb service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:24:07.410360 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f392be-8fcd-484a-ac1f-d6cca33544fb service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:24:07.416209 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039de3b3-6c1c-43c1-896a-ea37f9aa85ca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9b82080a73d4c4a8ec08b0e67d6d11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:24:07.416803 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039de3b3-6c1c-43c1-896a-ea37f9aa85ca service swift] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:24:07.417083 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039de3b3-6c1c-43c1-896a-ea37f9aa85ca service swift] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:24:07.417351 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039de3b3-6c1c-43c1-896a-ea37f9aa85ca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:24:07.425645 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039de3b3-6c1c-43c1-896a-ea37f9aa85ca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:24:07.427266 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039de3b3-6c1c-43c1-896a-ea37f9aa85ca service swift] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:24:07.435276 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1207/2400] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Aug 31 05:24:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 05:24:17.367609 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:24:17.432258 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=12 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:24:41.723018 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d04a9ae-a205-479e-b7b4-45d1225b1240 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:24:41.724784 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3d04a9ae-a205-479e-b7b4-45d1225b1240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:24:41.742755 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d04a9ae-a205-479e-b7b4-45d1225b1240 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:24:41.749585 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eedb6c-69b8-4eb4-bacc-80b428880f83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:24:41.750219 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eedb6c-69b8-4eb4-bacc-80b428880f83 service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:24:41.750492 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eedb6c-69b8-4eb4-bacc-80b428880f83 service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:24:41.750758 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eedb6c-69b8-4eb4-bacc-80b428880f83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:24:41.752640 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-12eedb6c-69b8-4eb4-bacc-80b428880f83 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:24:41.813837 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eedb6c-69b8-4eb4-bacc-80b428880f83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:24:41.815412 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eedb6c-69b8-4eb4-bacc-80b428880f83 service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:24:41.825138 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1194/2401] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:24:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:24:51.820732 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=28 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:25:01.446476 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039de3b3-6c1c-43c1-896a-ea37f9aa85ca service swift] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:25:01.461168 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039de3b3-6c1c-43c1-896a-ea37f9aa85ca service swift] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:25:01.469065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab5bb5a-0be0-450d-91f6-04446ad82b51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:25:01.469657 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab5bb5a-0be0-450d-91f6-04446ad82b51 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:25:01.469938 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab5bb5a-0be0-450d-91f6-04446ad82b51 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:25:01.470231 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab5bb5a-0be0-450d-91f6-04446ad82b51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:25:01.472152 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7ab5bb5a-0be0-450d-91f6-04446ad82b51 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:25:01.530808 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab5bb5a-0be0-450d-91f6-04446ad82b51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:25:01.532399 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab5bb5a-0be0-450d-91f6-04446ad82b51 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:25:01.541560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1208/2402] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:25:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:25:11.537377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=28 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:25:16.305776 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eedb6c-69b8-4eb4-bacc-80b428880f83 service nova] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:25:16.316555 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eedb6c-69b8-4eb4-bacc-80b428880f83 service nova] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:25:16.321634 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dea8625-90a7-44c9-aec9-037ad5d628ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:25:16.322270 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea8625-90a7-44c9-aec9-037ad5d628ca service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:25:16.322546 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea8625-90a7-44c9-aec9-037ad5d628ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:25:16.322812 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea8625-90a7-44c9-aec9-037ad5d628ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:25:16.325195 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3dea8625-90a7-44c9-aec9-037ad5d628ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:25:16.338119 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dea8625-90a7-44c9-aec9-037ad5d628ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:25:16.339558 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dea8625-90a7-44c9-aec9-037ad5d628ca service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:25:16.347827 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1195/2403] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:25:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:25:16.517433 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b0d56-1f0a-4534-938c-d33fb2071a10 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:25:16.517968 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b0d56-1f0a-4534-938c-d33fb2071a10 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:25:16.517968 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b0d56-1f0a-4534-938c-d33fb2071a10 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:25:16.519372 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1209/2404] 10.222.0.36 () {54 vars in 921 bytes} [Wed Aug 31 05:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:25:16.527498 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e78f4d-731b-4cc3-8c02-7c84f097f1c8 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:25:16.527941 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e78f4d-731b-4cc3-8c02-7c84f097f1c8 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:25:16.528359 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e78f4d-731b-4cc3-8c02-7c84f097f1c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:25:16.549474 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-13e78f4d-731b-4cc3-8c02-7c84f097f1c8 None None] MFA Rules not processed for user `090e1409506542f6831e1f65d44aae74`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:25:16.550854 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-13e78f4d-731b-4cc3-8c02-7c84f097f1c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:25:16.557949 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1196/2405] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Aug 31 05:25:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:25:16.571042 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b0d56-1f0a-4534-938c-d33fb2071a10 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:25:16.573122 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-057b0d56-1f0a-4534-938c-d33fb2071a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:25:16.590748 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b0d56-1f0a-4534-938c-d33fb2071a10 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:25:16.595481 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bdb94b-f7c1-4c18-ab73-d24b5326e4df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:25:16.596101 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bdb94b-f7c1-4c18-ab73-d24b5326e4df service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:25:16.596383 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bdb94b-f7c1-4c18-ab73-d24b5326e4df service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:25:16.596652 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bdb94b-f7c1-4c18-ab73-d24b5326e4df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:25:16.605010 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bdb94b-f7c1-4c18-ab73-d24b5326e4df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:25:16.606485 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bdb94b-f7c1-4c18-ab73-d24b5326e4df service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:25:16.614588 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1210/2406] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:25:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:25:16.709261 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77073bd5-41c7-4d8b-8b60-acee6d71acca None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:25:16.709571 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77073bd5-41c7-4d8b-8b60-acee6d71acca None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:25:16.709898 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77073bd5-41c7-4d8b-8b60-acee6d71acca None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:25:16.711290 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1197/2407] 10.222.0.36 () {54 vars in 921 bytes} [Wed Aug 31 05:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:25:16.717651 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427c824-bc7d-44d7-a90f-0429021e76ea None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:25:16.718075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427c824-bc7d-44d7-a90f-0429021e76ea None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:25:16.718419 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427c824-bc7d-44d7-a90f-0429021e76ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:25:16.739452 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-5427c824-bc7d-44d7-a90f-0429021e76ea None None] MFA Rules not processed for user `f27665b1574a4db7b7a9771f7d775826`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:25:16.740793 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5427c824-bc7d-44d7-a90f-0429021e76ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:25:16.750778 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1211/2408] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Aug 31 05:25:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:25:16.761387 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77073bd5-41c7-4d8b-8b60-acee6d71acca None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:25:16.772491 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77073bd5-41c7-4d8b-8b60-acee6d71acca None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:25:16.776161 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c08304-fa07-4323-b38b-ed89a161bd7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:25:16.776755 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c08304-fa07-4323-b38b-ed89a161bd7e service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:25:16.777192 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c08304-fa07-4323-b38b-ed89a161bd7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:25:16.777458 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c08304-fa07-4323-b38b-ed89a161bd7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:25:16.788277 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c08304-fa07-4323-b38b-ed89a161bd7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:25:16.789828 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c08304-fa07-4323-b38b-ed89a161bd7e service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:25:16.800774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1198/2409] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:25:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:25:16.864288 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5427c824-bc7d-44d7-a90f-0429021e76ea None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:25:16.865808 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5427c824-bc7d-44d7-a90f-0429021e76ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:25:16.936410 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5427c824-bc7d-44d7-a90f-0429021e76ea None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:25:16.946679 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0f7bf-140e-47db-b684-b41457552c77 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd4387035099416789009f0ecb8ad282', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:25:16.947636 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0f7bf-140e-47db-b684-b41457552c77 service placement] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:25:16.948059 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0f7bf-140e-47db-b684-b41457552c77 service placement] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:25:16.948514 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0f7bf-140e-47db-b684-b41457552c77 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:25:16.950670 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-73e0f7bf-140e-47db-b684-b41457552c77 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:25:16.963233 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e0f7bf-140e-47db-b684-b41457552c77 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:25:16.964876 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e0f7bf-140e-47db-b684-b41457552c77 service placement] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:25:16.976225 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1212/2410] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:25:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:25:19.186391 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c08304-fa07-4323-b38b-ed89a161bd7e service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:25:19.201956 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c08304-fa07-4323-b38b-ed89a161bd7e service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:25:19.208818 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55fdf5c-7ba9-45b5-9d9e-f4a2e0b4e2cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:25:19.210370 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55fdf5c-7ba9-45b5-9d9e-f4a2e0b4e2cb service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:25:19.210733 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55fdf5c-7ba9-45b5-9d9e-f4a2e0b4e2cb service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:25:19.211073 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55fdf5c-7ba9-45b5-9d9e-f4a2e0b4e2cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:25:19.213016 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-d55fdf5c-7ba9-45b5-9d9e-f4a2e0b4e2cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:25:19.225890 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55fdf5c-7ba9-45b5-9d9e-f4a2e0b4e2cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:25:19.227890 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55fdf5c-7ba9-45b5-9d9e-f4a2e0b4e2cb service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:25:19.238824 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1199/2411] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:25:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:25:23.304527 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0f7bf-140e-47db-b684-b41457552c77 service placement] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:25:23.306235 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-73e0f7bf-140e-47db-b684-b41457552c77 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:25:23.321536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0f7bf-140e-47db-b684-b41457552c77 service placement] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:25:23.327486 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d099b-7548-48e4-92e2-39f907747a57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:25:23.328156 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d099b-7548-48e4-92e2-39f907747a57 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:25:23.328513 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d099b-7548-48e4-92e2-39f907747a57 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:25:23.328846 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d099b-7548-48e4-92e2-39f907747a57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:25:23.330940 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-923d099b-7548-48e4-92e2-39f907747a57 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:25:23.344081 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923d099b-7548-48e4-92e2-39f907747a57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:25:23.345845 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923d099b-7548-48e4-92e2-39f907747a57 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:25:23.357590 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1213/2412] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:25:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:25:28.819794 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d670d-3c52-4c2e-ade7-01d4a467c39d None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:25:28.820291 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d670d-3c52-4c2e-ade7-01d4a467c39d None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:25:28.820390 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d670d-3c52-4c2e-ade7-01d4a467c39d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:25:28.840391 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-ff7d670d-3c52-4c2e-ade7-01d4a467c39d None None] MFA Rules not processed for user `af50ca6347404cdf8eb213d27053af16`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:25:28.841581 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-ff7d670d-3c52-4c2e-ade7-01d4a467c39d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:25:28.848898 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1200/2413] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:25:28 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:25:29.451590 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde09f4-7d23-4642-a7ff-0e6cd9b19b5f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:25:29.452152 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde09f4-7d23-4642-a7ff-0e6cd9b19b5f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:25:29.452152 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde09f4-7d23-4642-a7ff-0e6cd9b19b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:25:29.472419 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-1cde09f4-7d23-4642-a7ff-0e6cd9b19b5f None None] MFA Rules not processed for user `806b011517ef4201a86c90472e0740d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:25:29.473724 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1cde09f4-7d23-4642-a7ff-0e6cd9b19b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:25:29.482176 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1214/2414] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:25:29 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:25:36.022240 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d670d-3c52-4c2e-ade7-01d4a467c39d None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:25:36.036921 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d670d-3c52-4c2e-ade7-01d4a467c39d None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:25:36.043924 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059bc0ed-c908-4161-bbf5-9d37fc5da5a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:25:36.044964 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059bc0ed-c908-4161-bbf5-9d37fc5da5a7 service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:25:36.045242 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059bc0ed-c908-4161-bbf5-9d37fc5da5a7 service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:25:36.045506 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059bc0ed-c908-4161-bbf5-9d37fc5da5a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:25:36.047699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-059bc0ed-c908-4161-bbf5-9d37fc5da5a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:25:36.064208 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059bc0ed-c908-4161-bbf5-9d37fc5da5a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:25:36.065820 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059bc0ed-c908-4161-bbf5-9d37fc5da5a7 service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:25:36.076626 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1201/2415] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:25:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:25:39.466938 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=50 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:25:46.072186 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=30 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:26:08.422806 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae81c72a-1656-4a11-ab70-7d13b15d826b None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:26:08.423375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae81c72a-1656-4a11-ab70-7d13b15d826b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:26:08.423570 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae81c72a-1656-4a11-ab70-7d13b15d826b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:26:08.447122 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ae81c72a-1656-4a11-ab70-7d13b15d826b None None] MFA Rules not processed for user `09bf6db88ce8494a937211a814c9ff84`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:26:08.449078 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ae81c72a-1656-4a11-ab70-7d13b15d826b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:26:08.457112 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1215/2416] 10.222.0.36 () {58 vars in 967 bytes} [Wed Aug 31 05:26:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:26:08.514096 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059bc0ed-c908-4161-bbf5-9d37fc5da5a7 service nova] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:26:08.527973 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059bc0ed-c908-4161-bbf5-9d37fc5da5a7 service nova] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:26:08.534666 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b320067a-5942-4075-aa09-6bc8574ee136 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9b82080a73d4c4a8ec08b0e67d6d11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:26:08.535437 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b320067a-5942-4075-aa09-6bc8574ee136 service swift] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:26:08.535828 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b320067a-5942-4075-aa09-6bc8574ee136 service swift] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:26:08.536196 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b320067a-5942-4075-aa09-6bc8574ee136 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:26:08.561113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b320067a-5942-4075-aa09-6bc8574ee136 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:26:08.565560 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b320067a-5942-4075-aa09-6bc8574ee136 service swift] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:26:08.576314 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1202/2417] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Aug 31 05:26:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 05:26:11.722301 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af8d32-e9ff-4a8a-84cd-323bf395dccd None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:26:11.722919 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af8d32-e9ff-4a8a-84cd-323bf395dccd None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:26:11.722919 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af8d32-e9ff-4a8a-84cd-323bf395dccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:26:11.743884 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-85af8d32-e9ff-4a8a-84cd-323bf395dccd None None] MFA Rules not processed for user `4b07cff5bc3a4e68b62c4c5d3e798732`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:26:11.745369 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-85af8d32-e9ff-4a8a-84cd-323bf395dccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:26:11.753440 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1216/2418] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:26:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:26:12.341932 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b320067a-5942-4075-aa09-6bc8574ee136 service swift] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:26:12.353466 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b320067a-5942-4075-aa09-6bc8574ee136 service swift] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:26:12.357730 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b52237-0e56-4468-bc79-e992471c5d21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:26:12.358405 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b52237-0e56-4468-bc79-e992471c5d21 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:26:12.358701 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b52237-0e56-4468-bc79-e992471c5d21 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:26:12.358990 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b52237-0e56-4468-bc79-e992471c5d21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:26:12.360879 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-89b52237-0e56-4468-bc79-e992471c5d21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:26:12.371458 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b52237-0e56-4468-bc79-e992471c5d21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:26:12.372924 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b52237-0e56-4468-bc79-e992471c5d21 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:26:12.382463 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1203/2419] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:26:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:26:12.713055 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af8d32-e9ff-4a8a-84cd-323bf395dccd None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:26:12.714646 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-85af8d32-e9ff-4a8a-84cd-323bf395dccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:26:12.735298 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af8d32-e9ff-4a8a-84cd-323bf395dccd None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:26:12.743330 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf225e-c303-4744-a3ab-d8b115f21610 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd4387035099416789009f0ecb8ad282', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:26:12.744210 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf225e-c303-4744-a3ab-d8b115f21610 service placement] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:26:12.744758 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf225e-c303-4744-a3ab-d8b115f21610 service placement] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:26:12.745266 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf225e-c303-4744-a3ab-d8b115f21610 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:26:12.747292 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d1bf225e-c303-4744-a3ab-d8b115f21610 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:26:12.767465 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf225e-c303-4744-a3ab-d8b115f21610 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:26:12.774320 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf225e-c303-4744-a3ab-d8b115f21610 service placement] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:26:12.795143 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1217/2420] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:26:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:26:16.450518 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b52237-0e56-4468-bc79-e992471c5d21 service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:26:16.465472 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b52237-0e56-4468-bc79-e992471c5d21 service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:26:16.470739 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f2f32-1442-4263-aae7-0873e47e8f20 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd4387035099416789009f0ecb8ad282', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:26:16.471373 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f2f32-1442-4263-aae7-0873e47e8f20 service placement] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:26:16.471654 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f2f32-1442-4263-aae7-0873e47e8f20 service placement] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:26:16.472313 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f2f32-1442-4263-aae7-0873e47e8f20 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:26:16.475007 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-055f2f32-1442-4263-aae7-0873e47e8f20 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:26:16.488250 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055f2f32-1442-4263-aae7-0873e47e8f20 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:26:16.490047 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055f2f32-1442-4263-aae7-0873e47e8f20 service placement] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:26:16.501777 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1204/2421] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:26:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:26:16.573339 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf225e-c303-4744-a3ab-d8b115f21610 service placement] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:26:16.575002 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d1bf225e-c303-4744-a3ab-d8b115f21610 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:26:16.663239 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf225e-c303-4744-a3ab-d8b115f21610 service placement] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:26:16.669399 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da87b958-0f1e-4665-8c6d-f651b65cfeed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:26:16.670181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87b958-0f1e-4665-8c6d-f651b65cfeed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:26:16.670569 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87b958-0f1e-4665-8c6d-f651b65cfeed admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:26:16.670914 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87b958-0f1e-4665-8c6d-f651b65cfeed admin admin] PATH_INFO: `/v3/users/57600fcb074741e782f0eda3ae51e6dd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:26:16.672137 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da87b958-0f1e-4665-8c6d-f651b65cfeed admin admin] RBAC: Authorizing `identity:delete_user(user_id=57600fcb074741e782f0eda3ae51e6dd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:26:16.674196 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da87b958-0f1e-4665-8c6d-f651b65cfeed admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:26:16.731216 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-da87b958-0f1e-4665-8c6d-f651b65cfeed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57600fcb074741e782f0eda3ae51e6dd'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:26:16.739433 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-da87b958-0f1e-4665-8c6d-f651b65cfeed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57600fcb074741e782f0eda3ae51e6dd'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:26:16.750191 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-da87b958-0f1e-4665-8c6d-f651b65cfeed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57600fcb074741e782f0eda3ae51e6dd'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:26:16.751031 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-da87b958-0f1e-4665-8c6d-f651b65cfeed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57600fcb074741e782f0eda3ae51e6dd'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:26:16.764539 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-da87b958-0f1e-4665-8c6d-f651b65cfeed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57600fcb074741e782f0eda3ae51e6dd'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:26:16.774089 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1218/2422] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:26:16 2022] DELETE /identity/v3/users/57600fcb074741e782f0eda3ae51e6dd => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:26:16.952015 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f2f32-1442-4263-aae7-0873e47e8f20 service placement] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:26:16.971617 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f2f32-1442-4263-aae7-0873e47e8f20 service placement] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:26:17.036554 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca90bb12-cada-455e-9a9b-4298851b9503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:26:17.037189 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90bb12-cada-455e-9a9b-4298851b9503 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:26:17.037469 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90bb12-cada-455e-9a9b-4298851b9503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:26:17.037741 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90bb12-cada-455e-9a9b-4298851b9503 admin admin] PATH_INFO: `/v3/projects/4ad69415b75d4d259c8cd31e05b2a505` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:26:17.038914 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca90bb12-cada-455e-9a9b-4298851b9503 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ad69415b75d4d259c8cd31e05b2a505)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:26:17.040903 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca90bb12-cada-455e-9a9b-4298851b9503 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:26:17.089221 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ca90bb12-cada-455e-9a9b-4298851b9503 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ad69415b75d4d259c8cd31e05b2a505', 'request_id': 'req-ca90bb12-cada-455e-9a9b-4298851b9503'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:26:17.095787 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ca90bb12-cada-455e-9a9b-4298851b9503 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ad69415b75d4d259c8cd31e05b2a505', 'request_id': 'req-ca90bb12-cada-455e-9a9b-4298851b9503'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:26:17.101773 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ca90bb12-cada-455e-9a9b-4298851b9503 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ad69415b75d4d259c8cd31e05b2a505', 'request_id': 'req-ca90bb12-cada-455e-9a9b-4298851b9503'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:26:17.115271 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ca90bb12-cada-455e-9a9b-4298851b9503 admin admin] The token cache is being invalidate because project 4ad69415b75d4d259c8cd31e05b2a505 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:26:17.115577 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ca90bb12-cada-455e-9a9b-4298851b9503 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:26:17.116195 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ca90bb12-cada-455e-9a9b-4298851b9503 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:26:17.117678 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1205/2423] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:26:16 2022] DELETE /identity/v3/projects/4ad69415b75d4d259c8cd31e05b2a505 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:26:17.121892 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da87b958-0f1e-4665-8c6d-f651b65cfeed admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:26:17.123298 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-da87b958-0f1e-4665-8c6d-f651b65cfeed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:26:17.135986 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da87b958-0f1e-4665-8c6d-f651b65cfeed admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:26:17.183110 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:26:17.193980 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b8a5f-4db6-4d24-865a-1abb47e1e433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:26:17.194640 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b8a5f-4db6-4d24-865a-1abb47e1e433 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:26:17.194918 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b8a5f-4db6-4d24-865a-1abb47e1e433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:26:17.195184 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b8a5f-4db6-4d24-865a-1abb47e1e433 admin admin] PATH_INFO: `/v3/users/627e8ed7cbfe48fe8ed21bb0333ea28c` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:26:17.196315 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60b8a5f-4db6-4d24-865a-1abb47e1e433 admin admin] RBAC: Authorizing `identity:delete_user(user_id=627e8ed7cbfe48fe8ed21bb0333ea28c)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:26:17.198289 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60b8a5f-4db6-4d24-865a-1abb47e1e433 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:26:17.250109 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-f60b8a5f-4db6-4d24-865a-1abb47e1e433 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '627e8ed7cbfe48fe8ed21bb0333ea28c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:26:17.260063 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-f60b8a5f-4db6-4d24-865a-1abb47e1e433 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '627e8ed7cbfe48fe8ed21bb0333ea28c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:26:17.267745 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-f60b8a5f-4db6-4d24-865a-1abb47e1e433 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '627e8ed7cbfe48fe8ed21bb0333ea28c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:26:17.268636 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-f60b8a5f-4db6-4d24-865a-1abb47e1e433 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '627e8ed7cbfe48fe8ed21bb0333ea28c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:26:17.281334 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-f60b8a5f-4db6-4d24-865a-1abb47e1e433 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '627e8ed7cbfe48fe8ed21bb0333ea28c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:26:17.287417 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1219/2424] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:26:17 2022] DELETE /identity/v3/users/627e8ed7cbfe48fe8ed21bb0333ea28c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:26:17.472471 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca90bb12-cada-455e-9a9b-4298851b9503 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:26:17.474080 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-ca90bb12-cada-455e-9a9b-4298851b9503 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:26:17.487884 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca90bb12-cada-455e-9a9b-4298851b9503 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:26:17.537576 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:26:17.550861 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521332db-4870-4895-942f-b2247280159c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:26:17.551529 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521332db-4870-4895-942f-b2247280159c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:26:17.552020 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521332db-4870-4895-942f-b2247280159c admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:26:17.552284 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521332db-4870-4895-942f-b2247280159c admin admin] PATH_INFO: `/v3/projects/554993f9094d481d98a5cdff1b1387fa` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:26:17.553424 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521332db-4870-4895-942f-b2247280159c admin admin] RBAC: Authorizing `identity:delete_project(project_id=554993f9094d481d98a5cdff1b1387fa)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:26:17.555680 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521332db-4870-4895-942f-b2247280159c admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:26:17.602597 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-521332db-4870-4895-942f-b2247280159c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '554993f9094d481d98a5cdff1b1387fa', 'request_id': 'req-521332db-4870-4895-942f-b2247280159c'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:26:17.609311 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-521332db-4870-4895-942f-b2247280159c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '554993f9094d481d98a5cdff1b1387fa', 'request_id': 'req-521332db-4870-4895-942f-b2247280159c'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:26:17.614901 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-521332db-4870-4895-942f-b2247280159c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '554993f9094d481d98a5cdff1b1387fa', 'request_id': 'req-521332db-4870-4895-942f-b2247280159c'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:26:17.628595 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-521332db-4870-4895-942f-b2247280159c admin admin] The token cache is being invalidate because project 554993f9094d481d98a5cdff1b1387fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:26:17.628872 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-521332db-4870-4895-942f-b2247280159c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:26:17.629520 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-521332db-4870-4895-942f-b2247280159c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:26:17.631152 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1206/2425] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:26:17 2022] DELETE /identity/v3/projects/554993f9094d481d98a5cdff1b1387fa => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:26:27.283872 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:26:27.625424 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=28,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:26:43.776162 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b8a5f-4db6-4d24-865a-1abb47e1e433 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:26:43.778462 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f60b8a5f-4db6-4d24-865a-1abb47e1e433 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:26:43.871764 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b8a5f-4db6-4d24-865a-1abb47e1e433 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:26:43.920522 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bfd0ab-2bc4-47e5-86e8-46a096d8dbdc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:26:43.921056 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bfd0ab-2bc4-47e5-86e8-46a096d8dbdc service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:26:43.921332 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bfd0ab-2bc4-47e5-86e8-46a096d8dbdc service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:26:43.921602 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bfd0ab-2bc4-47e5-86e8-46a096d8dbdc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:26:43.923576 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-09bfd0ab-2bc4-47e5-86e8-46a096d8dbdc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:26:43.998557 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bfd0ab-2bc4-47e5-86e8-46a096d8dbdc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:26:44.000056 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bfd0ab-2bc4-47e5-86e8-46a096d8dbdc service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:26:44.009313 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1220/2426] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:26:43 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:26:54.005176 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=87 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:26:58.230410 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521332db-4870-4895-942f-b2247280159c admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:26:58.231981 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-521332db-4870-4895-942f-b2247280159c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:26:58.353576 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521332db-4870-4895-942f-b2247280159c admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:26:58.359090 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a4bb9d-ae0d-4ae6-8a6d-215eaa999da9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd4387035099416789009f0ecb8ad282', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:26:58.359739 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a4bb9d-ae0d-4ae6-8a6d-215eaa999da9 service placement] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:26:58.360148 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a4bb9d-ae0d-4ae6-8a6d-215eaa999da9 service placement] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:26:58.360432 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a4bb9d-ae0d-4ae6-8a6d-215eaa999da9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:26:58.362279 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-d8a4bb9d-ae0d-4ae6-8a6d-215eaa999da9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:26:58.374506 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a4bb9d-ae0d-4ae6-8a6d-215eaa999da9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:26:58.376171 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a4bb9d-ae0d-4ae6-8a6d-215eaa999da9 service placement] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:26:58.386903 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1207/2427] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:26:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:27:04.062409 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bfd0ab-2bc4-47e5-86e8-46a096d8dbdc service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:27:04.064072 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-09bfd0ab-2bc4-47e5-86e8-46a096d8dbdc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:27:04.153982 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bfd0ab-2bc4-47e5-86e8-46a096d8dbdc service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:27:04.161435 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d490684a-d248-4288-8c08-743f6bfcf3e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:27:04.162496 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490684a-d248-4288-8c08-743f6bfcf3e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:27:04.163001 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490684a-d248-4288-8c08-743f6bfcf3e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:27:04.163361 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490684a-d248-4288-8c08-743f6bfcf3e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:27:04.165490 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d490684a-d248-4288-8c08-743f6bfcf3e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:27:04.255454 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d490684a-d248-4288-8c08-743f6bfcf3e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:27:04.256880 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d490684a-d248-4288-8c08-743f6bfcf3e2 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:27:04.276128 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1221/2428] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:27:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:27:07.491945 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a4bb9d-ae0d-4ae6-8a6d-215eaa999da9 service placement] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:27:07.498330 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-d8a4bb9d-ae0d-4ae6-8a6d-215eaa999da9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:27:07.512382 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a4bb9d-ae0d-4ae6-8a6d-215eaa999da9 service placement] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:27:07.519564 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9354e9-4416-4072-803c-1925d6c55f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:27:07.520211 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9354e9-4416-4072-803c-1925d6c55f9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:27:07.520490 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9354e9-4416-4072-803c-1925d6c55f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:27:07.520759 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9354e9-4416-4072-803c-1925d6c55f9a admin admin] PATH_INFO: `/v3/users/806b011517ef4201a86c90472e0740d6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:27:07.522030 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9354e9-4416-4072-803c-1925d6c55f9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=806b011517ef4201a86c90472e0740d6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:27:07.523985 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9354e9-4416-4072-803c-1925d6c55f9a admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:27:07.573375 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-4a9354e9-4416-4072-803c-1925d6c55f9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '806b011517ef4201a86c90472e0740d6'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:27:07.580798 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-4a9354e9-4416-4072-803c-1925d6c55f9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '806b011517ef4201a86c90472e0740d6'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:27:07.581515 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-4a9354e9-4416-4072-803c-1925d6c55f9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '806b011517ef4201a86c90472e0740d6'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:27:07.589938 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-4a9354e9-4416-4072-803c-1925d6c55f9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '806b011517ef4201a86c90472e0740d6'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:27:07.590591 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-4a9354e9-4416-4072-803c-1925d6c55f9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '806b011517ef4201a86c90472e0740d6'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:27:07.607687 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1208/2429] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:27:07 2022] DELETE /identity/v3/users/806b011517ef4201a86c90472e0740d6 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:27:07.784549 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d490684a-d248-4288-8c08-743f6bfcf3e2 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:27:07.796689 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d490684a-d248-4288-8c08-743f6bfcf3e2 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:27:07.854366 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f90d89-a421-44a9-8218-e06be4b8ee1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:27:07.854911 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f90d89-a421-44a9-8218-e06be4b8ee1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:27:07.855184 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f90d89-a421-44a9-8218-e06be4b8ee1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:27:07.855446 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f90d89-a421-44a9-8218-e06be4b8ee1c admin admin] PATH_INFO: `/v3/projects/597fddc7874a4133987f79807674561f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:27:07.856555 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f90d89-a421-44a9-8218-e06be4b8ee1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=597fddc7874a4133987f79807674561f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:27:07.858524 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f90d89-a421-44a9-8218-e06be4b8ee1c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:27:07.860692 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:27:07.906627 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-43f90d89-a421-44a9-8218-e06be4b8ee1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '597fddc7874a4133987f79807674561f', 'request_id': 'req-43f90d89-a421-44a9-8218-e06be4b8ee1c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:27:07.913019 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-43f90d89-a421-44a9-8218-e06be4b8ee1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '597fddc7874a4133987f79807674561f', 'request_id': 'req-43f90d89-a421-44a9-8218-e06be4b8ee1c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:27:07.921543 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-43f90d89-a421-44a9-8218-e06be4b8ee1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '597fddc7874a4133987f79807674561f', 'request_id': 'req-43f90d89-a421-44a9-8218-e06be4b8ee1c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:27:07.935422 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-43f90d89-a421-44a9-8218-e06be4b8ee1c admin admin] The token cache is being invalidate because project 597fddc7874a4133987f79807674561f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:27:07.935700 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-43f90d89-a421-44a9-8218-e06be4b8ee1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:27:07.936324 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-43f90d89-a421-44a9-8218-e06be4b8ee1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:27:07.937802 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1222/2430] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:27:07 2022] DELETE /identity/v3/projects/597fddc7874a4133987f79807674561f => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:27:07.942791 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418981b-bdd6-48eb-a2f2-884cc2d6ed98 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:27:07.943069 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418981b-bdd6-48eb-a2f2-884cc2d6ed98 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:27:07.943366 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418981b-bdd6-48eb-a2f2-884cc2d6ed98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:27:07.966260 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:27:08.006135 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-4418981b-bdd6-48eb-a2f2-884cc2d6ed98 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:27:08.007336 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-4418981b-bdd6-48eb-a2f2-884cc2d6ed98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:27:08.067840 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1209/2431] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:27:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:27:08.075515 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f96b8-e104-4b6a-8da0-53d5bd319d4a None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:27:08.075839 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f96b8-e104-4b6a-8da0-53d5bd319d4a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:27:08.076126 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f96b8-e104-4b6a-8da0-53d5bd319d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:27:08.098253 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-6a0f96b8-e104-4b6a-8da0-53d5bd319d4a None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:27:08.099495 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6a0f96b8-e104-4b6a-8da0-53d5bd319d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:27:08.114141 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1223/2432] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:27:08.115449 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418981b-bdd6-48eb-a2f2-884cc2d6ed98 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:27:08.125854 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418981b-bdd6-48eb-a2f2-884cc2d6ed98 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:27:08.131145 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a1835a-17f0-4f5f-a825-8d6650859716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:27:08.131797 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a1835a-17f0-4f5f-a825-8d6650859716 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:27:08.132050 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a1835a-17f0-4f5f-a825-8d6650859716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:27:08.132320 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a1835a-17f0-4f5f-a825-8d6650859716 admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:27:08.132969 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a1835a-17f0-4f5f-a825-8d6650859716 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:27:08.133265 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a1835a-17f0-4f5f-a825-8d6650859716 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:27:08.134916 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a1835a-17f0-4f5f-a825-8d6650859716 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:27:08.147018 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1210/2433] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:27:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:27:08.151414 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0f96b8-e104-4b6a-8da0-53d5bd319d4a None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:27:08.161001 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0f96b8-e104-4b6a-8da0-53d5bd319d4a None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:27:08.165935 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d7803-7f0c-4d4e-9105-48a38793e5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:27:08.166525 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d7803-7f0c-4d4e-9105-48a38793e5d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:27:08.166797 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d7803-7f0c-4d4e-9105-48a38793e5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:27:08.167061 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d7803-7f0c-4d4e-9105-48a38793e5d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:27:08.167696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3d7803-7f0c-4d4e-9105-48a38793e5d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:27:08.169612 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3d7803-7f0c-4d4e-9105-48a38793e5d6 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:27:08.200524 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1224/2434] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:27:08 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:27:08.205216 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a1835a-17f0-4f5f-a825-8d6650859716 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:27:08.215838 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a1835a-17f0-4f5f-a825-8d6650859716 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:27:08.273414 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2fb0be-f24d-4a3f-a02e-5e3705e3d516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:27:08.274045 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2fb0be-f24d-4a3f-a02e-5e3705e3d516 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:27:08.274326 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2fb0be-f24d-4a3f-a02e-5e3705e3d516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:27:08.274597 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2fb0be-f24d-4a3f-a02e-5e3705e3d516 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:27:08.275230 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2fb0be-f24d-4a3f-a02e-5e3705e3d516 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:27:08.277181 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2fb0be-f24d-4a3f-a02e-5e3705e3d516 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:27:08.299065 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1211/2435] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:27:08 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:27:08.303271 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d7803-7f0c-4d4e-9105-48a38793e5d6 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:27:08.313367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d7803-7f0c-4d4e-9105-48a38793e5d6 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:27:08.318189 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ecb7a3-a7b4-4056-8e1f-d2956bbe855f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:27:08.318786 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ecb7a3-a7b4-4056-8e1f-d2956bbe855f admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:27:08.319061 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ecb7a3-a7b4-4056-8e1f-d2956bbe855f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:27:08.319326 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ecb7a3-a7b4-4056-8e1f-d2956bbe855f admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:27:08.320060 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ecb7a3-a7b4-4056-8e1f-d2956bbe855f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:27:08.321661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ecb7a3-a7b4-4056-8e1f-d2956bbe855f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:27:08.331754 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1225/2436] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:27:08.336202 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2fb0be-f24d-4a3f-a02e-5e3705e3d516 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:27:08.347189 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2fb0be-f24d-4a3f-a02e-5e3705e3d516 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:27:08.353108 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b0964-be1b-439e-98c8-9d5dd1509519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:27:08.353699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b0964-be1b-439e-98c8-9d5dd1509519 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:27:08.353977 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b0964-be1b-439e-98c8-9d5dd1509519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:27:08.354358 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b0964-be1b-439e-98c8-9d5dd1509519 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:27:08.354990 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820b0964-be1b-439e-98c8-9d5dd1509519 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:27:08.356605 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820b0964-be1b-439e-98c8-9d5dd1509519 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:27:08.367991 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1212/2437] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:27:08.371826 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ecb7a3-a7b4-4056-8e1f-d2956bbe855f admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:27:08.381983 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ecb7a3-a7b4-4056-8e1f-d2956bbe855f admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:27:08.387012 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48845dd0-29a1-4b1f-a789-82b7dcb696b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:27:08.387645 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48845dd0-29a1-4b1f-a789-82b7dcb696b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:27:08.387922 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48845dd0-29a1-4b1f-a789-82b7dcb696b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:27:08.388185 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48845dd0-29a1-4b1f-a789-82b7dcb696b2 admin admin] PATH_INFO: `/v3/projects/cb164f6efb1d4047b9a5ca77f2ff44a7/users/b0f728c7e4074084be1a433ae14879b8/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:27:08.403156 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48845dd0-29a1-4b1f-a789-82b7dcb696b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb164f6efb1d4047b9a5ca77f2ff44a7, user_id=b0f728c7e4074084be1a433ae14879b8, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:27:08.405847 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48845dd0-29a1-4b1f-a789-82b7dcb696b2 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:27:08.421774 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1226/2438] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:27:08 2022] PUT /identity/v3/projects/cb164f6efb1d4047b9a5ca77f2ff44a7/users/b0f728c7e4074084be1a433ae14879b8/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:27:08.426861 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd46fcc5-4f0f-46eb-b122-a8b6614f0b3e None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:27:08.427153 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd46fcc5-4f0f-46eb-b122-a8b6614f0b3e None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:27:08.427428 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd46fcc5-4f0f-46eb-b122-a8b6614f0b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:27:08.447624 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-dd46fcc5-4f0f-46eb-b122-a8b6614f0b3e None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:27:08.448851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-dd46fcc5-4f0f-46eb-b122-a8b6614f0b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:27:08.505407 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:27:08.520275 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1213/2439] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:27:08.530714 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48845dd0-29a1-4b1f-a789-82b7dcb696b2 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:27:08.533634 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-48845dd0-29a1-4b1f-a789-82b7dcb696b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:27:08.593618 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48845dd0-29a1-4b1f-a789-82b7dcb696b2 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:27:08.599250 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3d768-cba1-4541-93b4-a23167599105 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:27:08.599869 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3d768-cba1-4541-93b4-a23167599105 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:27:08.600142 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3d768-cba1-4541-93b4-a23167599105 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:27:08.600409 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3d768-cba1-4541-93b4-a23167599105 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:27:08.610681 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d3d768-cba1-4541-93b4-a23167599105 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:27:08.612121 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d3d768-cba1-4541-93b4-a23167599105 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:27:08.621742 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1227/2440] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:27:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:27:12.645641 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e7b5f-3eed-43d4-bfe6-e0102b5c8229 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:27:12.646169 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e7b5f-3eed-43d4-bfe6-e0102b5c8229 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:27:12.646272 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e7b5f-3eed-43d4-bfe6-e0102b5c8229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:27:12.683649 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-927e7b5f-3eed-43d4-bfe6-e0102b5c8229 None None] MFA Rules not processed for user `b0f728c7e4074084be1a433ae14879b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:27:12.684873 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-927e7b5f-3eed-43d4-bfe6-e0102b5c8229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:27:12.756374 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1214/2441] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:27:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:27:12.774195 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3d768-cba1-4541-93b4-a23167599105 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:27:12.779460 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-55d3d768-cba1-4541-93b4-a23167599105 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:27:12.832193 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:27:12.915879 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3d768-cba1-4541-93b4-a23167599105 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:27:12.921928 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa0c6e-2fed-42c3-9ed0-0fe9f39b5cfc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:27:12.922594 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa0c6e-2fed-42c3-9ed0-0fe9f39b5cfc service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:27:12.922872 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa0c6e-2fed-42c3-9ed0-0fe9f39b5cfc service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:27:12.923143 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa0c6e-2fed-42c3-9ed0-0fe9f39b5cfc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:27:12.932336 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa0c6e-2fed-42c3-9ed0-0fe9f39b5cfc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:27:12.933955 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa0c6e-2fed-42c3-9ed0-0fe9f39b5cfc service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:27:12.943889 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1228/2442] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:27:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:27:14.594465 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb69a7-1962-4262-acae-906ef5515b6a None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:27:14.595040 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb69a7-1962-4262-acae-906ef5515b6a None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:27:14.595040 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb69a7-1962-4262-acae-906ef5515b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:27:14.615680 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-cbdb69a7-1962-4262-acae-906ef5515b6a None None] MFA Rules not processed for user `b0f728c7e4074084be1a433ae14879b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:27:14.616866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-cbdb69a7-1962-4262-acae-906ef5515b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:27:14.623959 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1215/2443] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:27:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:27:22.939639 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=32 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:27:24.610062 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=38 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:27:25.866652 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa0c6e-2fed-42c3-9ed0-0fe9f39b5cfc service nova] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:27:25.876583 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa0c6e-2fed-42c3-9ed0-0fe9f39b5cfc service nova] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:27:25.880995 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca506456-c82d-4ebf-af13-e8446c4f29b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:27:25.881859 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca506456-c82d-4ebf-af13-e8446c4f29b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:27:25.882703 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca506456-c82d-4ebf-af13-e8446c4f29b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:27:25.883029 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca506456-c82d-4ebf-af13-e8446c4f29b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:27:25.885299 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ca506456-c82d-4ebf-af13-e8446c4f29b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:27:25.899814 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca506456-c82d-4ebf-af13-e8446c4f29b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:27:25.901284 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca506456-c82d-4ebf-af13-e8446c4f29b8 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:27:25.913545 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1229/2444] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:27:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:27:31.173720 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23901efd-476d-42df-8aeb-33b493b6b489 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:27:31.174341 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23901efd-476d-42df-8aeb-33b493b6b489 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:27:31.174341 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23901efd-476d-42df-8aeb-33b493b6b489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:27:31.203495 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-23901efd-476d-42df-8aeb-33b493b6b489 None None] MFA Rules not processed for user `09bf6db88ce8494a937211a814c9ff84`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:27:31.204669 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-23901efd-476d-42df-8aeb-33b493b6b489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:27:31.269049 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1216/2445] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Aug 31 05:27:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:27:31.320797 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca506456-c82d-4ebf-af13-e8446c4f29b8 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:27:31.322282 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ca506456-c82d-4ebf-af13-e8446c4f29b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:27:31.397300 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca506456-c82d-4ebf-af13-e8446c4f29b8 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:27:31.405261 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac1bf42-750c-421a-b91f-cb0eb997821a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9b82080a73d4c4a8ec08b0e67d6d11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:27:31.406051 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1bf42-750c-421a-b91f-cb0eb997821a service swift] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:27:31.408011 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1bf42-750c-421a-b91f-cb0eb997821a service swift] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:27:31.408011 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1bf42-750c-421a-b91f-cb0eb997821a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:27:31.423194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac1bf42-750c-421a-b91f-cb0eb997821a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:27:31.425136 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac1bf42-750c-421a-b91f-cb0eb997821a service swift] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:27:31.435621 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1230/2446] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Aug 31 05:27:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 05:27:34.133522 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23901efd-476d-42df-8aeb-33b493b6b489 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:27:34.135015 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-23901efd-476d-42df-8aeb-33b493b6b489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:27:34.153838 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23901efd-476d-42df-8aeb-33b493b6b489 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:27:34.159197 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52599a61-6569-413c-9bd0-e562e10054af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:27:34.159844 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52599a61-6569-413c-9bd0-e562e10054af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:27:34.160120 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52599a61-6569-413c-9bd0-e562e10054af admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:27:34.160397 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52599a61-6569-413c-9bd0-e562e10054af admin admin] PATH_INFO: `/v3/users/a0063b9fa5dc4c3789db536f99a211fc` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:27:34.161528 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52599a61-6569-413c-9bd0-e562e10054af admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0063b9fa5dc4c3789db536f99a211fc)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:27:34.163646 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52599a61-6569-413c-9bd0-e562e10054af admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:27:34.218236 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-52599a61-6569-413c-9bd0-e562e10054af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0063b9fa5dc4c3789db536f99a211fc'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:27:34.226288 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-52599a61-6569-413c-9bd0-e562e10054af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0063b9fa5dc4c3789db536f99a211fc'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:27:34.227093 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-52599a61-6569-413c-9bd0-e562e10054af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0063b9fa5dc4c3789db536f99a211fc'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:27:34.238420 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-52599a61-6569-413c-9bd0-e562e10054af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0063b9fa5dc4c3789db536f99a211fc'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:27:34.239273 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-52599a61-6569-413c-9bd0-e562e10054af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0063b9fa5dc4c3789db536f99a211fc'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:27:34.264015 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1217/2447] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:27:34 2022] DELETE /identity/v3/users/a0063b9fa5dc4c3789db536f99a211fc => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:27:34.452257 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac1bf42-750c-421a-b91f-cb0eb997821a service swift] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:27:34.480752 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac1bf42-750c-421a-b91f-cb0eb997821a service swift] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:27:34.563586 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180eee33-a7ac-4d96-9864-86956dcbed4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:27:34.564376 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180eee33-a7ac-4d96-9864-86956dcbed4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:27:34.564679 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180eee33-a7ac-4d96-9864-86956dcbed4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:27:34.565043 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180eee33-a7ac-4d96-9864-86956dcbed4b admin admin] PATH_INFO: `/v3/projects/ae98273d8d0040b39ed83c8b7aee74a6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:27:34.567886 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180eee33-a7ac-4d96-9864-86956dcbed4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae98273d8d0040b39ed83c8b7aee74a6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:27:34.569866 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180eee33-a7ac-4d96-9864-86956dcbed4b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:27:34.631997 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-180eee33-a7ac-4d96-9864-86956dcbed4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae98273d8d0040b39ed83c8b7aee74a6', 'request_id': 'req-180eee33-a7ac-4d96-9864-86956dcbed4b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:27:34.646817 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-180eee33-a7ac-4d96-9864-86956dcbed4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae98273d8d0040b39ed83c8b7aee74a6', 'request_id': 'req-180eee33-a7ac-4d96-9864-86956dcbed4b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:27:34.680867 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-180eee33-a7ac-4d96-9864-86956dcbed4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae98273d8d0040b39ed83c8b7aee74a6', 'request_id': 'req-180eee33-a7ac-4d96-9864-86956dcbed4b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:27:34.693929 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-180eee33-a7ac-4d96-9864-86956dcbed4b admin admin] The token cache is being invalidate because project ae98273d8d0040b39ed83c8b7aee74a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:27:34.694468 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-180eee33-a7ac-4d96-9864-86956dcbed4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:27:34.695282 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-180eee33-a7ac-4d96-9864-86956dcbed4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:27:34.701820 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1231/2448] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:27:34 2022] DELETE /identity/v3/projects/ae98273d8d0040b39ed83c8b7aee74a6 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:27:34.706078 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52599a61-6569-413c-9bd0-e562e10054af admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:27:34.707919 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-52599a61-6569-413c-9bd0-e562e10054af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:27:34.729086 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52599a61-6569-413c-9bd0-e562e10054af admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:27:34.839634 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a60ef7-1b40-4069-aae8-7bd30b806dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:27:34.840519 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a60ef7-1b40-4069-aae8-7bd30b806dd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:27:34.840977 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a60ef7-1b40-4069-aae8-7bd30b806dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:27:34.841490 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a60ef7-1b40-4069-aae8-7bd30b806dd5 admin admin] PATH_INFO: `/v3/users/1228fd3ec0df43ed8723d95ff00cbf5d` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:27:34.842890 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a60ef7-1b40-4069-aae8-7bd30b806dd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1228fd3ec0df43ed8723d95ff00cbf5d)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:27:34.845028 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a60ef7-1b40-4069-aae8-7bd30b806dd5 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:27:34.858610 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:27:34.918087 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-35a60ef7-1b40-4069-aae8-7bd30b806dd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1228fd3ec0df43ed8723d95ff00cbf5d'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:27:34.926358 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-35a60ef7-1b40-4069-aae8-7bd30b806dd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1228fd3ec0df43ed8723d95ff00cbf5d'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:27:34.927261 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-35a60ef7-1b40-4069-aae8-7bd30b806dd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1228fd3ec0df43ed8723d95ff00cbf5d'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:27:34.936894 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-35a60ef7-1b40-4069-aae8-7bd30b806dd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1228fd3ec0df43ed8723d95ff00cbf5d'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:27:34.937672 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-35a60ef7-1b40-4069-aae8-7bd30b806dd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1228fd3ec0df43ed8723d95ff00cbf5d'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:27:34.962984 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1218/2449] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:27:34 2022] DELETE /identity/v3/users/1228fd3ec0df43ed8723d95ff00cbf5d => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:27:35.160320 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180eee33-a7ac-4d96-9864-86956dcbed4b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:27:35.162181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-180eee33-a7ac-4d96-9864-86956dcbed4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:27:35.191649 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180eee33-a7ac-4d96-9864-86956dcbed4b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:27:35.197590 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:27:35.271526 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e72cf3f-9604-446b-abdd-e3ef84063104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:27:35.272256 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e72cf3f-9604-446b-abdd-e3ef84063104 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:27:35.272549 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e72cf3f-9604-446b-abdd-e3ef84063104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:27:35.272806 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e72cf3f-9604-446b-abdd-e3ef84063104 admin admin] PATH_INFO: `/v3/projects/3ba29a5baed640e599b49e898ac5a1cf` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:27:35.274300 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e72cf3f-9604-446b-abdd-e3ef84063104 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ba29a5baed640e599b49e898ac5a1cf)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:27:35.276337 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e72cf3f-9604-446b-abdd-e3ef84063104 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:27:35.329736 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6e72cf3f-9604-446b-abdd-e3ef84063104 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ba29a5baed640e599b49e898ac5a1cf', 'request_id': 'req-6e72cf3f-9604-446b-abdd-e3ef84063104'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:27:35.337016 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6e72cf3f-9604-446b-abdd-e3ef84063104 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ba29a5baed640e599b49e898ac5a1cf', 'request_id': 'req-6e72cf3f-9604-446b-abdd-e3ef84063104'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:27:35.346691 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6e72cf3f-9604-446b-abdd-e3ef84063104 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ba29a5baed640e599b49e898ac5a1cf', 'request_id': 'req-6e72cf3f-9604-446b-abdd-e3ef84063104'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:27:35.359566 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6e72cf3f-9604-446b-abdd-e3ef84063104 admin admin] The token cache is being invalidate because project 3ba29a5baed640e599b49e898ac5a1cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:27:35.359969 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6e72cf3f-9604-446b-abdd-e3ef84063104 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:27:35.360711 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6e72cf3f-9604-446b-abdd-e3ef84063104 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:27:35.374452 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1232/2450] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:27:35 2022] DELETE /identity/v3/projects/3ba29a5baed640e599b49e898ac5a1cf => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:27:44.958995 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:DELETE=10,keystone:SELECT=22,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:27:45.356788 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:27:46.538635 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f631f51f-f35d-4750-9225-b1955ee65b2a None None] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:27:46.539281 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f631f51f-f35d-4750-9225-b1955ee65b2a None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:27:46.539281 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f631f51f-f35d-4750-9225-b1955ee65b2a None None] PATH_INFO: `/` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:27:46.540994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1219/2451] 10.222.0.36 () {54 vars in 921 bytes} [Wed Aug 31 05:27:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 05:27:46.546865 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6ddcd-9e93-4324-a6f8-92a4684c6ac0 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:27:46.547364 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6ddcd-9e93-4324-a6f8-92a4684c6ac0 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:27:46.547483 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6ddcd-9e93-4324-a6f8-92a4684c6ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:27:46.569254 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-9ec6ddcd-9e93-4324-a6f8-92a4684c6ac0 None None] MFA Rules not processed for user `f27665b1574a4db7b7a9771f7d775826`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:27:46.570522 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9ec6ddcd-9e93-4324-a6f8-92a4684c6ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:27:46.654230 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1233/2452] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Aug 31 05:27:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:27:46.666066 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f631f51f-f35d-4750-9225-b1955ee65b2a None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:27:46.667931 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-f631f51f-f35d-4750-9225-b1955ee65b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:27:46.736116 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f631f51f-f35d-4750-9225-b1955ee65b2a None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:27:46.739679 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad491183-383a-4367-8600-0b99fa7ce007 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c08e6024b0c8420aa274653c2a916edd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:27:46.740353 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad491183-383a-4367-8600-0b99fa7ce007 service glance] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:27:46.740659 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad491183-383a-4367-8600-0b99fa7ce007 service glance] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:27:46.740931 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad491183-383a-4367-8600-0b99fa7ce007 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:27:46.753488 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad491183-383a-4367-8600-0b99fa7ce007 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:27:46.754972 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad491183-383a-4367-8600-0b99fa7ce007 service glance] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:27:46.765730 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1220/2453] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:27:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:27:48.739520 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cdf39f-dd6d-4084-8a75-88f9d7146506 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:27:48.739973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cdf39f-dd6d-4084-8a75-88f9d7146506 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:27:48.740086 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cdf39f-dd6d-4084-8a75-88f9d7146506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:27:48.761795 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b0cdf39f-dd6d-4084-8a75-88f9d7146506 None None] MFA Rules not processed for user `09bf6db88ce8494a937211a814c9ff84`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:27:48.763029 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b0cdf39f-dd6d-4084-8a75-88f9d7146506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:27:48.818840 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1234/2454] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Aug 31 05:27:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:27:48.830944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad491183-383a-4367-8600-0b99fa7ce007 service glance] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:27:48.832448 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-ad491183-383a-4367-8600-0b99fa7ce007 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:27:48.915723 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad491183-383a-4367-8600-0b99fa7ce007 service glance] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:27:48.921915 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c0eed3-e36a-4ee4-aaee-fe6d162c3c44 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9b82080a73d4c4a8ec08b0e67d6d11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:27:48.922573 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c0eed3-e36a-4ee4-aaee-fe6d162c3c44 service swift] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:27:48.922934 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c0eed3-e36a-4ee4-aaee-fe6d162c3c44 service swift] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:27:48.923108 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c0eed3-e36a-4ee4-aaee-fe6d162c3c44 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:27:48.932156 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c0eed3-e36a-4ee4-aaee-fe6d162c3c44 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:27:48.933764 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c0eed3-e36a-4ee4-aaee-fe6d162c3c44 service swift] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:27:48.942455 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1221/2455] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Aug 31 05:27:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 05:27:58.812039 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=56 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:27:58.938712 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=58 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:28:10.210363 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec6fbb-9c9c-4867-baee-acdb0de0367a None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:10.210913 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec6fbb-9c9c-4867-baee-acdb0de0367a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:10.210913 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec6fbb-9c9c-4867-baee-acdb0de0367a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:10.231740 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-2eec6fbb-9c9c-4867-baee-acdb0de0367a None None] MFA Rules not processed for user `09bf6db88ce8494a937211a814c9ff84`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:28:10.232987 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2eec6fbb-9c9c-4867-baee-acdb0de0367a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:10.240938 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1235/2456] 10.222.0.36 () {58 vars in 967 bytes} [Wed Aug 31 05:28:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:28:10.254257 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c0eed3-e36a-4ee4-aaee-fe6d162c3c44 service swift] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:10.265236 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c0eed3-e36a-4ee4-aaee-fe6d162c3c44 service swift] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:10.270807 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec33cf1-8136-4c36-bed3-5782ebce0134 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9b82080a73d4c4a8ec08b0e67d6d11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:10.271448 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec33cf1-8136-4c36-bed3-5782ebce0134 service swift] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:10.271751 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec33cf1-8136-4c36-bed3-5782ebce0134 service swift] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:10.272015 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec33cf1-8136-4c36-bed3-5782ebce0134 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:10.282073 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec33cf1-8136-4c36-bed3-5782ebce0134 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:10.283299 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec33cf1-8136-4c36-bed3-5782ebce0134 service swift] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:10.291945 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1222/2457] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Aug 31 05:28:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 05:28:20.226266 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:28:20.288536 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=6 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:28:29.314213 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eec6fbb-9c9c-4867-baee-acdb0de0367a None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:29.315760 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2eec6fbb-9c9c-4867-baee-acdb0de0367a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:29.374330 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eec6fbb-9c9c-4867-baee-acdb0de0367a None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:29.378475 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7917346f-aa9c-4d09-b9df-0600c36fb9e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:29.379110 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917346f-aa9c-4d09-b9df-0600c36fb9e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:29.379381 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917346f-aa9c-4d09-b9df-0600c36fb9e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:29.379645 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917346f-aa9c-4d09-b9df-0600c36fb9e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:29.381499 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7917346f-aa9c-4d09-b9df-0600c36fb9e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:29.446653 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7917346f-aa9c-4d09-b9df-0600c36fb9e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:29.448112 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7917346f-aa9c-4d09-b9df-0600c36fb9e6 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:29.458134 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1236/2458] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:28:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:28:32.702810 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec33cf1-8136-4c36-bed3-5782ebce0134 service swift] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:32.704306 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-eec33cf1-8136-4c36-bed3-5782ebce0134 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:32.724388 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec33cf1-8136-4c36-bed3-5782ebce0134 service swift] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:32.730245 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b309795-2d32-4454-bb96-b5e5280c1a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:32.730869 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b309795-2d32-4454-bb96-b5e5280c1a29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:32.731140 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b309795-2d32-4454-bb96-b5e5280c1a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:32.731822 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b309795-2d32-4454-bb96-b5e5280c1a29 admin admin] PATH_INFO: `/v3/users/4b07cff5bc3a4e68b62c4c5d3e798732` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:32.748847 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b309795-2d32-4454-bb96-b5e5280c1a29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b07cff5bc3a4e68b62c4c5d3e798732)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:32.750914 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b309795-2d32-4454-bb96-b5e5280c1a29 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:32.800812 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-7b309795-2d32-4454-bb96-b5e5280c1a29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b07cff5bc3a4e68b62c4c5d3e798732'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:32.807842 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-7b309795-2d32-4454-bb96-b5e5280c1a29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b07cff5bc3a4e68b62c4c5d3e798732'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:32.808502 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-7b309795-2d32-4454-bb96-b5e5280c1a29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b07cff5bc3a4e68b62c4c5d3e798732'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:32.817456 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-7b309795-2d32-4454-bb96-b5e5280c1a29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b07cff5bc3a4e68b62c4c5d3e798732'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:32.818162 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-7b309795-2d32-4454-bb96-b5e5280c1a29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b07cff5bc3a4e68b62c4c5d3e798732'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:32.837816 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1223/2459] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:28:32 2022] DELETE /identity/v3/users/4b07cff5bc3a4e68b62c4c5d3e798732 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:28:33.021069 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7917346f-aa9c-4d09-b9df-0600c36fb9e6 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:33.031831 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7917346f-aa9c-4d09-b9df-0600c36fb9e6 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:33.091378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc7608-1511-4614-a15a-8dc3922ba373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:33.091986 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc7608-1511-4614-a15a-8dc3922ba373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:33.092262 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc7608-1511-4614-a15a-8dc3922ba373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:33.092525 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc7608-1511-4614-a15a-8dc3922ba373 admin admin] PATH_INFO: `/v3/projects/14058365e648441d96bb725c33f80923` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:33.104063 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc7608-1511-4614-a15a-8dc3922ba373 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14058365e648441d96bb725c33f80923)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:33.105986 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc7608-1511-4614-a15a-8dc3922ba373 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:33.150196 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:28:33.154463 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a9bc7608-1511-4614-a15a-8dc3922ba373 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14058365e648441d96bb725c33f80923', 'request_id': 'req-a9bc7608-1511-4614-a15a-8dc3922ba373'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:33.161336 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a9bc7608-1511-4614-a15a-8dc3922ba373 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14058365e648441d96bb725c33f80923', 'request_id': 'req-a9bc7608-1511-4614-a15a-8dc3922ba373'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:33.169950 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a9bc7608-1511-4614-a15a-8dc3922ba373 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14058365e648441d96bb725c33f80923', 'request_id': 'req-a9bc7608-1511-4614-a15a-8dc3922ba373'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:33.181094 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a9bc7608-1511-4614-a15a-8dc3922ba373 admin admin] The token cache is being invalidate because project 14058365e648441d96bb725c33f80923 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:28:33.181377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a9bc7608-1511-4614-a15a-8dc3922ba373 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:33.182103 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a9bc7608-1511-4614-a15a-8dc3922ba373 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:33.183655 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1237/2460] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:28:33 2022] DELETE /identity/v3/projects/14058365e648441d96bb725c33f80923 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:28:33.188614 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d1906-04cb-442a-8d8a-d0ca247eccc7 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:33.188902 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d1906-04cb-442a-8d8a-d0ca247eccc7 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:33.189205 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d1906-04cb-442a-8d8a-d0ca247eccc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:33.215360 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-b88d1906-04cb-442a-8d8a-d0ca247eccc7 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:28:33.216937 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b88d1906-04cb-442a-8d8a-d0ca247eccc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:33.292552 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1224/2461] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:28:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:28:33.300780 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439151e0-5567-479e-a689-053687e767b0 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:33.301066 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439151e0-5567-479e-a689-053687e767b0 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:33.301336 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439151e0-5567-479e-a689-053687e767b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:33.322631 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-439151e0-5567-479e-a689-053687e767b0 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:28:33.323842 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-439151e0-5567-479e-a689-053687e767b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:33.333137 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1238/2462] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:28:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:28:33.337473 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88d1906-04cb-442a-8d8a-d0ca247eccc7 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:33.347664 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88d1906-04cb-442a-8d8a-d0ca247eccc7 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:33.352576 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb4bf6d-fbf9-4c2f-b397-8065f73f0b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:33.353254 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4bf6d-fbf9-4c2f-b397-8065f73f0b4f admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:33.353529 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4bf6d-fbf9-4c2f-b397-8065f73f0b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:33.353793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4bf6d-fbf9-4c2f-b397-8065f73f0b4f admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:33.354461 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb4bf6d-fbf9-4c2f-b397-8065f73f0b4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:28:33.354759 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb4bf6d-fbf9-4c2f-b397-8065f73f0b4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:33.356423 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb4bf6d-fbf9-4c2f-b397-8065f73f0b4f admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:33.368144 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1225/2463] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:28:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:28:33.372595 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439151e0-5567-479e-a689-053687e767b0 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:33.382562 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439151e0-5567-479e-a689-053687e767b0 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:33.388698 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b834a0-c191-4ec2-892f-0b66be9469a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:33.389304 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b834a0-c191-4ec2-892f-0b66be9469a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:33.389579 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b834a0-c191-4ec2-892f-0b66be9469a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:33.389834 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b834a0-c191-4ec2-892f-0b66be9469a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:33.390510 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b834a0-c191-4ec2-892f-0b66be9469a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:33.392418 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b834a0-c191-4ec2-892f-0b66be9469a2 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:33.432110 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1239/2464] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:28:33 2022] POST /identity/v3/projects => generated 365 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:28:33.438593 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb4bf6d-fbf9-4c2f-b397-8065f73f0b4f admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:33.451915 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb4bf6d-fbf9-4c2f-b397-8065f73f0b4f admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:33.496215 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:28:33.510456 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa073e6-2bc5-4bdb-9186-65fd4494045f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:33.511064 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa073e6-2bc5-4bdb-9186-65fd4494045f admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:33.511340 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa073e6-2bc5-4bdb-9186-65fd4494045f admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:33.511608 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa073e6-2bc5-4bdb-9186-65fd4494045f admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:33.512242 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa073e6-2bc5-4bdb-9186-65fd4494045f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:33.514217 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa073e6-2bc5-4bdb-9186-65fd4494045f admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:33.537738 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1226/2465] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:28:33 2022] POST /identity/v3/users => generated 283 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:28:33.542181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b834a0-c191-4ec2-892f-0b66be9469a2 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:33.552629 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b834a0-c191-4ec2-892f-0b66be9469a2 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:33.558261 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7779cdbd-6a99-468b-9e51-28ccbad2f410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:33.558859 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779cdbd-6a99-468b-9e51-28ccbad2f410 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:33.559168 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779cdbd-6a99-468b-9e51-28ccbad2f410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:33.559425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779cdbd-6a99-468b-9e51-28ccbad2f410 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:33.560066 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7779cdbd-6a99-468b-9e51-28ccbad2f410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:33.561661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7779cdbd-6a99-468b-9e51-28ccbad2f410 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:33.572193 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1240/2466] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:28:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:28:33.576266 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa073e6-2bc5-4bdb-9186-65fd4494045f admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:33.586370 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa073e6-2bc5-4bdb-9186-65fd4494045f admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:33.591574 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08160a8-8195-420f-a1df-13b12b8146e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:33.592197 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08160a8-8195-420f-a1df-13b12b8146e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:33.592470 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08160a8-8195-420f-a1df-13b12b8146e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:33.592735 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08160a8-8195-420f-a1df-13b12b8146e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:33.593378 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08160a8-8195-420f-a1df-13b12b8146e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:33.595433 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08160a8-8195-420f-a1df-13b12b8146e1 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:33.608171 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1227/2467] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:28:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:28:33.612257 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7779cdbd-6a99-468b-9e51-28ccbad2f410 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:33.622823 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7779cdbd-6a99-468b-9e51-28ccbad2f410 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:33.629392 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a647b8f2-c93a-469e-b48d-74930725385b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:33.630352 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647b8f2-c93a-469e-b48d-74930725385b admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:33.630633 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647b8f2-c93a-469e-b48d-74930725385b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:33.630960 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647b8f2-c93a-469e-b48d-74930725385b admin admin] PATH_INFO: `/v3/projects/decdb965018d47c3978408b2ab171a32/users/1ca6db4fb6be4ee6a371caf28f7c3f8a/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:33.650489 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a647b8f2-c93a-469e-b48d-74930725385b admin admin] RBAC: Authorizing `identity:create_grant(project_id=decdb965018d47c3978408b2ab171a32, user_id=1ca6db4fb6be4ee6a371caf28f7c3f8a, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:33.653694 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a647b8f2-c93a-469e-b48d-74930725385b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:33.669550 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1241/2468] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:28:33 2022] PUT /identity/v3/projects/decdb965018d47c3978408b2ab171a32/users/1ca6db4fb6be4ee6a371caf28f7c3f8a/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:28:33.680989 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63bbb8d-ecd5-46cb-82b4-5c24b5041d01 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:33.681397 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63bbb8d-ecd5-46cb-82b4-5c24b5041d01 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:33.681664 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63bbb8d-ecd5-46cb-82b4-5c24b5041d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:33.710150 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-f63bbb8d-ecd5-46cb-82b4-5c24b5041d01 None None] MFA Rules not processed for user `1ca6db4fb6be4ee6a371caf28f7c3f8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:28:33.711277 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-f63bbb8d-ecd5-46cb-82b4-5c24b5041d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:33.780279 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1228/2469] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:28:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:28:33.784943 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a647b8f2-c93a-469e-b48d-74930725385b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:33.847905 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a647b8f2-c93a-469e-b48d-74930725385b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:33.853001 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c880b0-6d08-46f0-a9e9-82fc3193514c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:33.853599 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c880b0-6d08-46f0-a9e9-82fc3193514c admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:33.853875 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c880b0-6d08-46f0-a9e9-82fc3193514c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:33.854271 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c880b0-6d08-46f0-a9e9-82fc3193514c admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:33.854905 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c880b0-6d08-46f0-a9e9-82fc3193514c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:33.856811 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c880b0-6d08-46f0-a9e9-82fc3193514c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:33.884297 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1242/2470] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:28:33 2022] POST /identity/v3/projects => generated 365 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:28:33.888593 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63bbb8d-ecd5-46cb-82b4-5c24b5041d01 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:33.899843 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63bbb8d-ecd5-46cb-82b4-5c24b5041d01 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:33.957394 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48541d1f-b0da-4644-8119-78b055a0af2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:33.957997 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48541d1f-b0da-4644-8119-78b055a0af2b admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:33.958294 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48541d1f-b0da-4644-8119-78b055a0af2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:33.958561 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48541d1f-b0da-4644-8119-78b055a0af2b admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:33.959195 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48541d1f-b0da-4644-8119-78b055a0af2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:33.961196 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48541d1f-b0da-4644-8119-78b055a0af2b admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:33.984374 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1229/2471] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:28:33 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:28:33.988460 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c880b0-6d08-46f0-a9e9-82fc3193514c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:33.998834 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c880b0-6d08-46f0-a9e9-82fc3193514c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:34.003666 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58446b60-f0bf-4e85-8dca-e5e774147442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:34.004256 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58446b60-f0bf-4e85-8dca-e5e774147442 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:34.004534 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58446b60-f0bf-4e85-8dca-e5e774147442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:34.004795 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58446b60-f0bf-4e85-8dca-e5e774147442 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:34.005432 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58446b60-f0bf-4e85-8dca-e5e774147442 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:34.007042 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58446b60-f0bf-4e85-8dca-e5e774147442 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:34.017526 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1243/2472] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:28:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 05:28:34.021536 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48541d1f-b0da-4644-8119-78b055a0af2b admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:34.031496 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48541d1f-b0da-4644-8119-78b055a0af2b admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:34.036457 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f381f581-835a-4dda-8481-e6545c808a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:34.037096 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381f581-835a-4dda-8481-e6545c808a86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:34.037372 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381f581-835a-4dda-8481-e6545c808a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:34.037639 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381f581-835a-4dda-8481-e6545c808a86 admin admin] PATH_INFO: `/v3/projects/3743b00c367c41bdafc05e56c26b6369/users/7945f2068f5240418a762d49dcbed19f/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:34.052372 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f381f581-835a-4dda-8481-e6545c808a86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3743b00c367c41bdafc05e56c26b6369, user_id=7945f2068f5240418a762d49dcbed19f, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:34.055117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f381f581-835a-4dda-8481-e6545c808a86 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:34.071699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1230/2473] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:28:34 2022] PUT /identity/v3/projects/3743b00c367c41bdafc05e56c26b6369/users/7945f2068f5240418a762d49dcbed19f/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:28:34.075775 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58446b60-f0bf-4e85-8dca-e5e774147442 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:34.085728 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58446b60-f0bf-4e85-8dca-e5e774147442 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:34.115292 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:28:34.143644 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc91bd8-90d8-469e-955a-ed4dc376ae15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:34.144276 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc91bd8-90d8-469e-955a-ed4dc376ae15 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:34.144552 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc91bd8-90d8-469e-955a-ed4dc376ae15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:34.144815 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc91bd8-90d8-469e-955a-ed4dc376ae15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:34.145453 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc91bd8-90d8-469e-955a-ed4dc376ae15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:34.147086 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc91bd8-90d8-469e-955a-ed4dc376ae15 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:34.157624 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1244/2474] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:28:34 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:28:34.162020 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f381f581-835a-4dda-8481-e6545c808a86 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:34.172831 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f381f581-835a-4dda-8481-e6545c808a86 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:34.177791 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8e2453-83f7-43c8-9ce5-0fe31e2d4d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:34.178464 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e2453-83f7-43c8-9ce5-0fe31e2d4d69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:34.178732 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e2453-83f7-43c8-9ce5-0fe31e2d4d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:34.178995 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e2453-83f7-43c8-9ce5-0fe31e2d4d69 admin admin] PATH_INFO: `/v3/projects/3743b00c367c41bdafc05e56c26b6369/users/7945f2068f5240418a762d49dcbed19f/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:34.180886 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8e2453-83f7-43c8-9ce5-0fe31e2d4d69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3743b00c367c41bdafc05e56c26b6369, user_id=7945f2068f5240418a762d49dcbed19f, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:34.183546 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8e2453-83f7-43c8-9ce5-0fe31e2d4d69 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:34.193787 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1231/2475] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:28:34 2022] PUT /identity/v3/projects/3743b00c367c41bdafc05e56c26b6369/users/7945f2068f5240418a762d49dcbed19f/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:28:34.204220 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842ebab-b019-43a6-a78d-18bc2cc056fa None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:34.204499 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842ebab-b019-43a6-a78d-18bc2cc056fa None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:34.204770 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842ebab-b019-43a6-a78d-18bc2cc056fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:34.234252 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-0842ebab-b019-43a6-a78d-18bc2cc056fa None None] MFA Rules not processed for user `7945f2068f5240418a762d49dcbed19f`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:28:34.235380 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0842ebab-b019-43a6-a78d-18bc2cc056fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:34.306107 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1245/2476] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:28:34 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:28:34.323168 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8e2453-83f7-43c8-9ce5-0fe31e2d4d69 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:34.325262 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-8a8e2453-83f7-43c8-9ce5-0fe31e2d4d69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:34.339102 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:28:34.407815 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8e2453-83f7-43c8-9ce5-0fe31e2d4d69 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:34.412254 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9812cd48-e72c-458e-8ac0-5494937ca692 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:34.412860 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812cd48-e72c-458e-8ac0-5494937ca692 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:34.413135 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812cd48-e72c-458e-8ac0-5494937ca692 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:34.413400 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812cd48-e72c-458e-8ac0-5494937ca692 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:34.423776 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9812cd48-e72c-458e-8ac0-5494937ca692 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:34.425237 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9812cd48-e72c-458e-8ac0-5494937ca692 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:34.434971 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1232/2477] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:28:34 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:28:34.872630 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0842ebab-b019-43a6-a78d-18bc2cc056fa None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:34.937697 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0842ebab-b019-43a6-a78d-18bc2cc056fa None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:34.942922 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cc131e-8b66-4d9e-a39c-b85ca0b8284c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:34.943549 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc131e-8b66-4d9e-a39c-b85ca0b8284c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:34.943829 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc131e-8b66-4d9e-a39c-b85ca0b8284c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:34.944094 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc131e-8b66-4d9e-a39c-b85ca0b8284c admin admin] PATH_INFO: `/v3/users/1ca6db4fb6be4ee6a371caf28f7c3f8a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:34.945250 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cc131e-8b66-4d9e-a39c-b85ca0b8284c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ca6db4fb6be4ee6a371caf28f7c3f8a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:34.947316 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cc131e-8b66-4d9e-a39c-b85ca0b8284c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:35.005155 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-95cc131e-8b66-4d9e-a39c-b85ca0b8284c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ca6db4fb6be4ee6a371caf28f7c3f8a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:35.011879 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-95cc131e-8b66-4d9e-a39c-b85ca0b8284c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ca6db4fb6be4ee6a371caf28f7c3f8a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:35.013507 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:28:35.019555 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-95cc131e-8b66-4d9e-a39c-b85ca0b8284c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ca6db4fb6be4ee6a371caf28f7c3f8a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:35.020228 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-95cc131e-8b66-4d9e-a39c-b85ca0b8284c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ca6db4fb6be4ee6a371caf28f7c3f8a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:35.031944 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-95cc131e-8b66-4d9e-a39c-b85ca0b8284c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ca6db4fb6be4ee6a371caf28f7c3f8a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:35.038079 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1246/2478] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:28:34 2022] DELETE /identity/v3/users/1ca6db4fb6be4ee6a371caf28f7c3f8a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:28:35.042933 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a103579-bf83-4560-8bf7-76c769f90fec None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:35.043470 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a103579-bf83-4560-8bf7-76c769f90fec None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:35.043606 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a103579-bf83-4560-8bf7-76c769f90fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:35.063227 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-7a103579-bf83-4560-8bf7-76c769f90fec None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:28:35.064424 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-7a103579-bf83-4560-8bf7-76c769f90fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:35.126826 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1233/2479] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:28:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:28:35.136632 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cc131e-8b66-4d9e-a39c-b85ca0b8284c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:35.138376 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-95cc131e-8b66-4d9e-a39c-b85ca0b8284c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:35.201392 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cc131e-8b66-4d9e-a39c-b85ca0b8284c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:35.205845 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc125e-427b-47e5-a8b2-a742216d533e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:35.206456 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc125e-427b-47e5-a8b2-a742216d533e service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:35.206760 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc125e-427b-47e5-a8b2-a742216d533e service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:35.207025 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc125e-427b-47e5-a8b2-a742216d533e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:35.216993 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cc125e-427b-47e5-a8b2-a742216d533e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:35.218928 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cc125e-427b-47e5-a8b2-a742216d533e service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:35.228977 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1247/2480] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:28:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 05:28:35.511560 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a103579-bf83-4560-8bf7-76c769f90fec None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:35.513106 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-7a103579-bf83-4560-8bf7-76c769f90fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:35.525196 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a103579-bf83-4560-8bf7-76c769f90fec None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:35.530086 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1253bd87-010f-4448-a15d-8f4696e82ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:35.530731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253bd87-010f-4448-a15d-8f4696e82ff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:35.530991 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253bd87-010f-4448-a15d-8f4696e82ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:35.531256 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253bd87-010f-4448-a15d-8f4696e82ff8 admin admin] PATH_INFO: `/v3/projects/decdb965018d47c3978408b2ab171a32` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:35.532399 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1253bd87-010f-4448-a15d-8f4696e82ff8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=decdb965018d47c3978408b2ab171a32)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:35.534308 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1253bd87-010f-4448-a15d-8f4696e82ff8 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:35.585181 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-1253bd87-010f-4448-a15d-8f4696e82ff8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'decdb965018d47c3978408b2ab171a32', 'request_id': 'req-1253bd87-010f-4448-a15d-8f4696e82ff8'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:35.591257 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-1253bd87-010f-4448-a15d-8f4696e82ff8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'decdb965018d47c3978408b2ab171a32', 'request_id': 'req-1253bd87-010f-4448-a15d-8f4696e82ff8'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:35.597120 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-1253bd87-010f-4448-a15d-8f4696e82ff8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'decdb965018d47c3978408b2ab171a32', 'request_id': 'req-1253bd87-010f-4448-a15d-8f4696e82ff8'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:35.611086 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-1253bd87-010f-4448-a15d-8f4696e82ff8 admin admin] The token cache is being invalidate because project decdb965018d47c3978408b2ab171a32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:28:35.611398 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-1253bd87-010f-4448-a15d-8f4696e82ff8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:35.612019 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-1253bd87-010f-4448-a15d-8f4696e82ff8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:35.613500 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1234/2481] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:28:35 2022] DELETE /identity/v3/projects/decdb965018d47c3978408b2ab171a32 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:28:35.617570 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc125e-427b-47e5-a8b2-a742216d533e service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:35.627498 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc125e-427b-47e5-a8b2-a742216d533e service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:35.683708 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3a5eb1-12c1-46a1-9e7d-8b1a7b1a29c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:35.684325 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a5eb1-12c1-46a1-9e7d-8b1a7b1a29c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:35.684604 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a5eb1-12c1-46a1-9e7d-8b1a7b1a29c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:35.684869 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a5eb1-12c1-46a1-9e7d-8b1a7b1a29c5 admin admin] PATH_INFO: `/v3/users/7945f2068f5240418a762d49dcbed19f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:35.686017 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3a5eb1-12c1-46a1-9e7d-8b1a7b1a29c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7945f2068f5240418a762d49dcbed19f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:35.687940 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3a5eb1-12c1-46a1-9e7d-8b1a7b1a29c5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:35.739043 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9e3a5eb1-12c1-46a1-9e7d-8b1a7b1a29c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7945f2068f5240418a762d49dcbed19f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:35.745936 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9e3a5eb1-12c1-46a1-9e7d-8b1a7b1a29c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7945f2068f5240418a762d49dcbed19f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:35.754386 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9e3a5eb1-12c1-46a1-9e7d-8b1a7b1a29c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7945f2068f5240418a762d49dcbed19f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:35.755017 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9e3a5eb1-12c1-46a1-9e7d-8b1a7b1a29c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7945f2068f5240418a762d49dcbed19f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:35.767368 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:28:35.768501 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9e3a5eb1-12c1-46a1-9e7d-8b1a7b1a29c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7945f2068f5240418a762d49dcbed19f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:35.774956 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1248/2482] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:28:35 2022] DELETE /identity/v3/users/7945f2068f5240418a762d49dcbed19f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:28:36.049806 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1253bd87-010f-4448-a15d-8f4696e82ff8 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:36.051385 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-1253bd87-010f-4448-a15d-8f4696e82ff8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:36.065248 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1253bd87-010f-4448-a15d-8f4696e82ff8 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:36.078368 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:28:36.126870 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0fa55-c16d-4966-8cd4-1d23d27cda5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:36.127453 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0fa55-c16d-4966-8cd4-1d23d27cda5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:36.127730 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0fa55-c16d-4966-8cd4-1d23d27cda5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:36.128023 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0fa55-c16d-4966-8cd4-1d23d27cda5a admin admin] PATH_INFO: `/v3/projects/3743b00c367c41bdafc05e56c26b6369` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:36.129207 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a0fa55-c16d-4966-8cd4-1d23d27cda5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3743b00c367c41bdafc05e56c26b6369)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:36.131336 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a0fa55-c16d-4966-8cd4-1d23d27cda5a admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:36.180206 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-04a0fa55-c16d-4966-8cd4-1d23d27cda5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3743b00c367c41bdafc05e56c26b6369', 'request_id': 'req-04a0fa55-c16d-4966-8cd4-1d23d27cda5a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:36.186546 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-04a0fa55-c16d-4966-8cd4-1d23d27cda5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3743b00c367c41bdafc05e56c26b6369', 'request_id': 'req-04a0fa55-c16d-4966-8cd4-1d23d27cda5a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:36.192646 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-04a0fa55-c16d-4966-8cd4-1d23d27cda5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3743b00c367c41bdafc05e56c26b6369', 'request_id': 'req-04a0fa55-c16d-4966-8cd4-1d23d27cda5a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:36.205539 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-04a0fa55-c16d-4966-8cd4-1d23d27cda5a admin admin] The token cache is being invalidate because project 3743b00c367c41bdafc05e56c26b6369 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:28:36.205822 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-04a0fa55-c16d-4966-8cd4-1d23d27cda5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:36.206461 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-04a0fa55-c16d-4966-8cd4-1d23d27cda5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:36.207941 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1235/2483] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:28:36 2022] DELETE /identity/v3/projects/3743b00c367c41bdafc05e56c26b6369 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:28:36.214087 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469b433-aa54-4fe2-add7-a7d87b7a4327 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:36.214554 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469b433-aa54-4fe2-add7-a7d87b7a4327 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:36.214649 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469b433-aa54-4fe2-add7-a7d87b7a4327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:36.234212 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a469b433-aa54-4fe2-add7-a7d87b7a4327 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:28:36.235403 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a469b433-aa54-4fe2-add7-a7d87b7a4327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:36.316571 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1249/2484] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:28:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:28:36.328244 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c38eb2f-ac97-421f-8456-8cfa738367c2 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:36.328540 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c38eb2f-ac97-421f-8456-8cfa738367c2 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:36.328810 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c38eb2f-ac97-421f-8456-8cfa738367c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:36.358242 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-3c38eb2f-ac97-421f-8456-8cfa738367c2 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:28:36.359454 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3c38eb2f-ac97-421f-8456-8cfa738367c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:36.368921 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1236/2485] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:28:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:28:36.373598 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a469b433-aa54-4fe2-add7-a7d87b7a4327 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:36.384223 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a469b433-aa54-4fe2-add7-a7d87b7a4327 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:36.390099 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4284d296-c968-4734-8a49-25e91b6b7905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:36.390698 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284d296-c968-4734-8a49-25e91b6b7905 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:36.390970 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284d296-c968-4734-8a49-25e91b6b7905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:36.391230 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284d296-c968-4734-8a49-25e91b6b7905 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:36.391881 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4284d296-c968-4734-8a49-25e91b6b7905 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:28:36.392177 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4284d296-c968-4734-8a49-25e91b6b7905 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:36.393776 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4284d296-c968-4734-8a49-25e91b6b7905 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:36.407344 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1250/2486] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:28:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:28:36.410862 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c38eb2f-ac97-421f-8456-8cfa738367c2 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:36.420580 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c38eb2f-ac97-421f-8456-8cfa738367c2 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:36.425520 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c66d33a-cefe-4d84-b192-c2542bea8e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:36.426179 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66d33a-cefe-4d84-b192-c2542bea8e0d admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:36.426458 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66d33a-cefe-4d84-b192-c2542bea8e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:36.426725 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66d33a-cefe-4d84-b192-c2542bea8e0d admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:36.427366 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c66d33a-cefe-4d84-b192-c2542bea8e0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:36.429270 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c66d33a-cefe-4d84-b192-c2542bea8e0d admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:36.458668 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1237/2487] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:28:36 2022] POST /identity/v3/projects => generated 399 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:28:36.463151 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4284d296-c968-4734-8a49-25e91b6b7905 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:36.473820 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4284d296-c968-4734-8a49-25e91b6b7905 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:36.531674 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8411c06f-7f65-4781-9525-097c1f1beabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:36.532274 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8411c06f-7f65-4781-9525-097c1f1beabc admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:36.532549 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8411c06f-7f65-4781-9525-097c1f1beabc admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:36.532809 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8411c06f-7f65-4781-9525-097c1f1beabc admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:36.533431 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8411c06f-7f65-4781-9525-097c1f1beabc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:36.535383 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8411c06f-7f65-4781-9525-097c1f1beabc admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:36.558153 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1251/2488] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:28:36 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:28:36.562589 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c66d33a-cefe-4d84-b192-c2542bea8e0d admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:36.573528 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c66d33a-cefe-4d84-b192-c2542bea8e0d admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:36.579359 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e9fac7-4006-4efa-878c-2bbb94607357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:36.579893 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e9fac7-4006-4efa-878c-2bbb94607357 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:36.580167 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e9fac7-4006-4efa-878c-2bbb94607357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:36.580430 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e9fac7-4006-4efa-878c-2bbb94607357 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:36.581097 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e9fac7-4006-4efa-878c-2bbb94607357 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:36.582765 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e9fac7-4006-4efa-878c-2bbb94607357 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:36.593452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1238/2489] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:28:36.598115 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8411c06f-7f65-4781-9525-097c1f1beabc admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:36.608885 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8411c06f-7f65-4781-9525-097c1f1beabc admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:36.614564 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b1daf7-66f5-4c13-acf8-9e55779738d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:36.615170 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b1daf7-66f5-4c13-acf8-9e55779738d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:36.615465 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b1daf7-66f5-4c13-acf8-9e55779738d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:36.615732 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b1daf7-66f5-4c13-acf8-9e55779738d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:36.616372 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b1daf7-66f5-4c13-acf8-9e55779738d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:36.617990 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b1daf7-66f5-4c13-acf8-9e55779738d7 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:36.629155 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1252/2490] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:28:36.633336 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e9fac7-4006-4efa-878c-2bbb94607357 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:36.643837 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e9fac7-4006-4efa-878c-2bbb94607357 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:36.649172 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf1f40-e19c-4be5-bce7-143cf85f7abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:36.649879 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf1f40-e19c-4be5-bce7-143cf85f7abd admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:36.650193 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf1f40-e19c-4be5-bce7-143cf85f7abd admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:36.650448 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf1f40-e19c-4be5-bce7-143cf85f7abd admin admin] PATH_INFO: `/v3/projects/7822b35c9fcd4f7da9e3db8ccc009e6c/users/b7971433d94340d280f318ad34e0356d/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:36.666112 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbf1f40-e19c-4be5-bce7-143cf85f7abd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7822b35c9fcd4f7da9e3db8ccc009e6c, user_id=b7971433d94340d280f318ad34e0356d, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:36.668773 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbf1f40-e19c-4be5-bce7-143cf85f7abd admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:36.683599 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1239/2491] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:28:36 2022] PUT /identity/v3/projects/7822b35c9fcd4f7da9e3db8ccc009e6c/users/b7971433d94340d280f318ad34e0356d/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:28:36.694116 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401478a-c009-4dbd-9112-8b9e8df37cef None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:36.694399 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401478a-c009-4dbd-9112-8b9e8df37cef None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:36.694664 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401478a-c009-4dbd-9112-8b9e8df37cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:36.723172 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-3401478a-c009-4dbd-9112-8b9e8df37cef None None] MFA Rules not processed for user `b7971433d94340d280f318ad34e0356d`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:28:36.724391 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3401478a-c009-4dbd-9112-8b9e8df37cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:36.772525 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:28:36.791460 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1253/2492] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:28:36 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:28:36.796250 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf1f40-e19c-4be5-bce7-143cf85f7abd admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:36.843257 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:28:36.861972 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf1f40-e19c-4be5-bce7-143cf85f7abd admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:36.866854 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887ba65-82e6-4c57-b44b-752d3f32d30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:36.867477 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887ba65-82e6-4c57-b44b-752d3f32d30f admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:36.867751 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887ba65-82e6-4c57-b44b-752d3f32d30f admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:36.868010 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887ba65-82e6-4c57-b44b-752d3f32d30f admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:36.868758 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6887ba65-82e6-4c57-b44b-752d3f32d30f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:36.870748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6887ba65-82e6-4c57-b44b-752d3f32d30f admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:36.899381 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1240/2493] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:28:36 2022] POST /identity/v3/projects => generated 401 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:28:36.903785 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3401478a-c009-4dbd-9112-8b9e8df37cef None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:36.915035 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3401478a-c009-4dbd-9112-8b9e8df37cef None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:36.972491 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07ca25c-1e1a-4732-943e-d1867cb6e1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:36.973104 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07ca25c-1e1a-4732-943e-d1867cb6e1a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:36.973383 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07ca25c-1e1a-4732-943e-d1867cb6e1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:36.973645 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07ca25c-1e1a-4732-943e-d1867cb6e1a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:36.974325 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07ca25c-1e1a-4732-943e-d1867cb6e1a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:36.976328 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07ca25c-1e1a-4732-943e-d1867cb6e1a6 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:37.002060 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1254/2494] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:28:36 2022] POST /identity/v3/users => generated 307 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:28:37.004625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887ba65-82e6-4c57-b44b-752d3f32d30f admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:37.015618 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887ba65-82e6-4c57-b44b-752d3f32d30f admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:37.021112 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10901fa-996e-4113-86e7-0c14f3ab8fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:37.021730 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10901fa-996e-4113-86e7-0c14f3ab8fcb admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:37.022072 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10901fa-996e-4113-86e7-0c14f3ab8fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:37.022330 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10901fa-996e-4113-86e7-0c14f3ab8fcb admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:37.022984 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10901fa-996e-4113-86e7-0c14f3ab8fcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:37.024636 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10901fa-996e-4113-86e7-0c14f3ab8fcb admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:37.035496 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1241/2495] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:28:37.039674 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07ca25c-1e1a-4732-943e-d1867cb6e1a6 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:37.050619 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07ca25c-1e1a-4732-943e-d1867cb6e1a6 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:37.056269 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9eac79-a091-462d-afff-2f04925f9766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:37.056934 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9eac79-a091-462d-afff-2f04925f9766 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:37.057195 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9eac79-a091-462d-afff-2f04925f9766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:37.057460 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9eac79-a091-462d-afff-2f04925f9766 admin admin] PATH_INFO: `/v3/projects/3eb1978e5f604a8abe579ce9122bfebd/users/3187265f23e84c81831eb2705b898220/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:37.072396 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9eac79-a091-462d-afff-2f04925f9766 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3eb1978e5f604a8abe579ce9122bfebd, user_id=3187265f23e84c81831eb2705b898220, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:37.075104 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9eac79-a091-462d-afff-2f04925f9766 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:37.089771 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1255/2496] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:28:37 2022] PUT /identity/v3/projects/3eb1978e5f604a8abe579ce9122bfebd/users/3187265f23e84c81831eb2705b898220/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:28:37.093884 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10901fa-996e-4113-86e7-0c14f3ab8fcb admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:37.105217 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10901fa-996e-4113-86e7-0c14f3ab8fcb admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:37.162090 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef1bdfa-a84d-4a58-8720-32b4b320ddad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:37.162631 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1bdfa-a84d-4a58-8720-32b4b320ddad admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:37.162913 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1bdfa-a84d-4a58-8720-32b4b320ddad admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:37.163186 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1bdfa-a84d-4a58-8720-32b4b320ddad admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:37.163850 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef1bdfa-a84d-4a58-8720-32b4b320ddad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:37.165503 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef1bdfa-a84d-4a58-8720-32b4b320ddad admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:37.175189 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1242/2497] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:28:37.179529 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9eac79-a091-462d-afff-2f04925f9766 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:37.189970 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9eac79-a091-462d-afff-2f04925f9766 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:37.194848 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc68d3-0eb0-48c9-b53d-ad8dd7986069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:37.195538 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc68d3-0eb0-48c9-b53d-ad8dd7986069 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:37.195813 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc68d3-0eb0-48c9-b53d-ad8dd7986069 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:37.196094 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc68d3-0eb0-48c9-b53d-ad8dd7986069 admin admin] PATH_INFO: `/v3/projects/3eb1978e5f604a8abe579ce9122bfebd/users/3187265f23e84c81831eb2705b898220/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:37.198225 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fc68d3-0eb0-48c9-b53d-ad8dd7986069 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3eb1978e5f604a8abe579ce9122bfebd, user_id=3187265f23e84c81831eb2705b898220, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:37.200855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fc68d3-0eb0-48c9-b53d-ad8dd7986069 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:37.211204 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1256/2498] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:28:37 2022] PUT /identity/v3/projects/3eb1978e5f604a8abe579ce9122bfebd/users/3187265f23e84c81831eb2705b898220/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:28:37.221917 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ac0fe-27d6-44f8-bc40-57241f9135cd None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:37.222230 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ac0fe-27d6-44f8-bc40-57241f9135cd None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:37.222500 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ac0fe-27d6-44f8-bc40-57241f9135cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:37.260661 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-b86ac0fe-27d6-44f8-bc40-57241f9135cd None None] MFA Rules not processed for user `3187265f23e84c81831eb2705b898220`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:28:37.261887 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b86ac0fe-27d6-44f8-bc40-57241f9135cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:37.337507 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1243/2499] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:28:37 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:28:37.353376 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc68d3-0eb0-48c9-b53d-ad8dd7986069 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:37.356464 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-39fc68d3-0eb0-48c9-b53d-ad8dd7986069 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:37.418922 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc68d3-0eb0-48c9-b53d-ad8dd7986069 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:37.423417 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9befa22c-c80c-41b2-8ae0-1eaa908b3d16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:37.424026 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9befa22c-c80c-41b2-8ae0-1eaa908b3d16 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:37.424300 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9befa22c-c80c-41b2-8ae0-1eaa908b3d16 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:37.424562 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9befa22c-c80c-41b2-8ae0-1eaa908b3d16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:37.434460 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9befa22c-c80c-41b2-8ae0-1eaa908b3d16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:37.435901 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9befa22c-c80c-41b2-8ae0-1eaa908b3d16 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:37.445326 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1257/2500] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:28:37 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:28:37.526119 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ac0fe-27d6-44f8-bc40-57241f9135cd None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:37.536174 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ac0fe-27d6-44f8-bc40-57241f9135cd None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:37.539936 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e02de-b5c7-4dca-b864-7cb0a9eb7fa6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:37.540556 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e02de-b5c7-4dca-b864-7cb0a9eb7fa6 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:37.540838 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e02de-b5c7-4dca-b864-7cb0a9eb7fa6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:37.541105 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e02de-b5c7-4dca-b864-7cb0a9eb7fa6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:37.575655 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:28:37.604452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36e02de-b5c7-4dca-b864-7cb0a9eb7fa6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:37.605928 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36e02de-b5c7-4dca-b864-7cb0a9eb7fa6 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:37.615538 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1244/2501] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:28:37 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:28:41.744259 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9befa22c-c80c-41b2-8ae0-1eaa908b3d16 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:41.792041 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:28:41.807640 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9befa22c-c80c-41b2-8ae0-1eaa908b3d16 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:41.812546 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc74a94-23b3-43c5-93e7-32bc2e9018ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:41.813167 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc74a94-23b3-43c5-93e7-32bc2e9018ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:41.813442 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc74a94-23b3-43c5-93e7-32bc2e9018ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:41.813707 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc74a94-23b3-43c5-93e7-32bc2e9018ff admin admin] PATH_INFO: `/v3/users/b7971433d94340d280f318ad34e0356d` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:41.814864 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc74a94-23b3-43c5-93e7-32bc2e9018ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7971433d94340d280f318ad34e0356d)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:41.816783 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc74a94-23b3-43c5-93e7-32bc2e9018ff admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:41.866470 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3fc74a94-23b3-43c5-93e7-32bc2e9018ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7971433d94340d280f318ad34e0356d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:41.873356 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3fc74a94-23b3-43c5-93e7-32bc2e9018ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7971433d94340d280f318ad34e0356d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:41.881529 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3fc74a94-23b3-43c5-93e7-32bc2e9018ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7971433d94340d280f318ad34e0356d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:41.882454 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3fc74a94-23b3-43c5-93e7-32bc2e9018ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7971433d94340d280f318ad34e0356d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:41.894842 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3fc74a94-23b3-43c5-93e7-32bc2e9018ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7971433d94340d280f318ad34e0356d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:41.900760 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1258/2502] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:28:41 2022] DELETE /identity/v3/users/b7971433d94340d280f318ad34e0356d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:28:41.905428 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c83c3-9a66-4bd2-9c62-b9bdbd9bb8f2 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:41.905908 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c83c3-9a66-4bd2-9c62-b9bdbd9bb8f2 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:41.906036 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c83c3-9a66-4bd2-9c62-b9bdbd9bb8f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:41.925718 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-426c83c3-9a66-4bd2-9c62-b9bdbd9bb8f2 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:28:41.927055 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-426c83c3-9a66-4bd2-9c62-b9bdbd9bb8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:41.990941 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1245/2503] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:28:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:28:42.000842 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc74a94-23b3-43c5-93e7-32bc2e9018ff admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:42.002405 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3fc74a94-23b3-43c5-93e7-32bc2e9018ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:42.059925 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc74a94-23b3-43c5-93e7-32bc2e9018ff admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:42.064128 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a8fc3-db1d-4ba2-bc34-dfe47a87ee39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:42.064737 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a8fc3-db1d-4ba2-bc34-dfe47a87ee39 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:42.065014 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a8fc3-db1d-4ba2-bc34-dfe47a87ee39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:42.065282 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a8fc3-db1d-4ba2-bc34-dfe47a87ee39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:42.074603 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5a8fc3-db1d-4ba2-bc34-dfe47a87ee39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:42.076032 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5a8fc3-db1d-4ba2-bc34-dfe47a87ee39 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:42.087437 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1259/2504] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:28:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:28:42.371091 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426c83c3-9a66-4bd2-9c62-b9bdbd9bb8f2 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:42.372684 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-426c83c3-9a66-4bd2-9c62-b9bdbd9bb8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:42.385625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426c83c3-9a66-4bd2-9c62-b9bdbd9bb8f2 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:42.390690 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bc7df7-267f-4aaf-897a-109b39839f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:42.391438 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc7df7-267f-4aaf-897a-109b39839f2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:42.391720 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc7df7-267f-4aaf-897a-109b39839f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:42.392085 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc7df7-267f-4aaf-897a-109b39839f2e admin admin] PATH_INFO: `/v3/projects/7822b35c9fcd4f7da9e3db8ccc009e6c` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:42.393232 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bc7df7-267f-4aaf-897a-109b39839f2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7822b35c9fcd4f7da9e3db8ccc009e6c)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:42.395204 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bc7df7-267f-4aaf-897a-109b39839f2e admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:42.443377 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-18bc7df7-267f-4aaf-897a-109b39839f2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7822b35c9fcd4f7da9e3db8ccc009e6c', 'request_id': 'req-18bc7df7-267f-4aaf-897a-109b39839f2e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:42.449672 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-18bc7df7-267f-4aaf-897a-109b39839f2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7822b35c9fcd4f7da9e3db8ccc009e6c', 'request_id': 'req-18bc7df7-267f-4aaf-897a-109b39839f2e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:42.456255 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-18bc7df7-267f-4aaf-897a-109b39839f2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7822b35c9fcd4f7da9e3db8ccc009e6c', 'request_id': 'req-18bc7df7-267f-4aaf-897a-109b39839f2e'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:42.469160 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-18bc7df7-267f-4aaf-897a-109b39839f2e admin admin] The token cache is being invalidate because project 7822b35c9fcd4f7da9e3db8ccc009e6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:28:42.469748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-18bc7df7-267f-4aaf-897a-109b39839f2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:42.470368 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-18bc7df7-267f-4aaf-897a-109b39839f2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:42.471960 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1246/2505] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:28:42 2022] DELETE /identity/v3/projects/7822b35c9fcd4f7da9e3db8ccc009e6c => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:28:42.475994 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a8fc3-db1d-4ba2-bc34-dfe47a87ee39 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:42.486147 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a8fc3-db1d-4ba2-bc34-dfe47a87ee39 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:42.545367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e47d691-c842-4f5f-b615-536292e7f548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:42.545991 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e47d691-c842-4f5f-b615-536292e7f548 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:42.546295 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e47d691-c842-4f5f-b615-536292e7f548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:42.546560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e47d691-c842-4f5f-b615-536292e7f548 admin admin] PATH_INFO: `/v3/users/3187265f23e84c81831eb2705b898220` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:42.547755 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e47d691-c842-4f5f-b615-536292e7f548 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3187265f23e84c81831eb2705b898220)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:42.549680 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e47d691-c842-4f5f-b615-536292e7f548 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:42.560404 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:28:42.600490 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8e47d691-c842-4f5f-b615-536292e7f548 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3187265f23e84c81831eb2705b898220'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:42.607259 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8e47d691-c842-4f5f-b615-536292e7f548 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3187265f23e84c81831eb2705b898220'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:42.616076 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8e47d691-c842-4f5f-b615-536292e7f548 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3187265f23e84c81831eb2705b898220'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:42.616651 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8e47d691-c842-4f5f-b615-536292e7f548 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3187265f23e84c81831eb2705b898220'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:42.628993 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8e47d691-c842-4f5f-b615-536292e7f548 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3187265f23e84c81831eb2705b898220'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:42.635100 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1260/2506] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:28:42 2022] DELETE /identity/v3/users/3187265f23e84c81831eb2705b898220 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:28:42.923640 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bc7df7-267f-4aaf-897a-109b39839f2e admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:42.926254 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-18bc7df7-267f-4aaf-897a-109b39839f2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:42.948088 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bc7df7-267f-4aaf-897a-109b39839f2e admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:43.010733 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db3fff7-0069-4112-9667-c52ece7199a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:43.011357 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3fff7-0069-4112-9667-c52ece7199a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:43.011634 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3fff7-0069-4112-9667-c52ece7199a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:43.011887 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3fff7-0069-4112-9667-c52ece7199a2 admin admin] PATH_INFO: `/v3/projects/3eb1978e5f604a8abe579ce9122bfebd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:43.013021 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db3fff7-0069-4112-9667-c52ece7199a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3eb1978e5f604a8abe579ce9122bfebd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:43.015056 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db3fff7-0069-4112-9667-c52ece7199a2 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:43.016524 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:28:43.065318 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-7db3fff7-0069-4112-9667-c52ece7199a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3eb1978e5f604a8abe579ce9122bfebd', 'request_id': 'req-7db3fff7-0069-4112-9667-c52ece7199a2'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:43.071630 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-7db3fff7-0069-4112-9667-c52ece7199a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3eb1978e5f604a8abe579ce9122bfebd', 'request_id': 'req-7db3fff7-0069-4112-9667-c52ece7199a2'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:43.077499 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-7db3fff7-0069-4112-9667-c52ece7199a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3eb1978e5f604a8abe579ce9122bfebd', 'request_id': 'req-7db3fff7-0069-4112-9667-c52ece7199a2'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:43.091100 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-7db3fff7-0069-4112-9667-c52ece7199a2 admin admin] The token cache is being invalidate because project 3eb1978e5f604a8abe579ce9122bfebd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:28:43.091383 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-7db3fff7-0069-4112-9667-c52ece7199a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:43.092012 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-7db3fff7-0069-4112-9667-c52ece7199a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:43.093517 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1247/2507] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:28:42 2022] DELETE /identity/v3/projects/3eb1978e5f604a8abe579ce9122bfebd => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:28:43.098831 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91245c73-c4f0-4aab-a012-1dddd2d28d6d None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:43.099616 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91245c73-c4f0-4aab-a012-1dddd2d28d6d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:43.099616 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91245c73-c4f0-4aab-a012-1dddd2d28d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:43.119115 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-91245c73-c4f0-4aab-a012-1dddd2d28d6d None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:28:43.120297 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-91245c73-c4f0-4aab-a012-1dddd2d28d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:43.190833 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1261/2508] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:28:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:28:43.203161 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf8215-4eaa-4b57-ab6c-bf01ffaacd07 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:43.203457 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf8215-4eaa-4b57-ab6c-bf01ffaacd07 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:43.203721 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf8215-4eaa-4b57-ab6c-bf01ffaacd07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:43.225778 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-c3bf8215-4eaa-4b57-ab6c-bf01ffaacd07 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:28:43.227590 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c3bf8215-4eaa-4b57-ab6c-bf01ffaacd07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:43.241706 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1248/2509] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:28:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:28:43.248485 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91245c73-c4f0-4aab-a012-1dddd2d28d6d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:43.260537 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91245c73-c4f0-4aab-a012-1dddd2d28d6d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:43.269803 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e268c20-c15d-4199-9fee-b1ddfb52c13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:43.270591 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e268c20-c15d-4199-9fee-b1ddfb52c13a admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:43.270865 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e268c20-c15d-4199-9fee-b1ddfb52c13a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:43.271169 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e268c20-c15d-4199-9fee-b1ddfb52c13a admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:43.272189 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e268c20-c15d-4199-9fee-b1ddfb52c13a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:28:43.272574 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e268c20-c15d-4199-9fee-b1ddfb52c13a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:43.275071 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e268c20-c15d-4199-9fee-b1ddfb52c13a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:43.287841 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1262/2510] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:28:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:28:43.292393 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bf8215-4eaa-4b57-ab6c-bf01ffaacd07 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:43.301655 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bf8215-4eaa-4b57-ab6c-bf01ffaacd07 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:43.306799 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b65495-0f9a-4302-bd9d-369e2771e685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:43.307378 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b65495-0f9a-4302-bd9d-369e2771e685 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:43.307648 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b65495-0f9a-4302-bd9d-369e2771e685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:43.307893 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b65495-0f9a-4302-bd9d-369e2771e685 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:43.308527 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b65495-0f9a-4302-bd9d-369e2771e685 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:43.310451 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b65495-0f9a-4302-bd9d-369e2771e685 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:43.341820 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1249/2511] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:28:43 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:28:43.347175 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e268c20-c15d-4199-9fee-b1ddfb52c13a admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:43.361482 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e268c20-c15d-4199-9fee-b1ddfb52c13a admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:43.417378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efbd953-1405-4c24-b075-51774fd521f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:43.417977 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efbd953-1405-4c24-b075-51774fd521f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:43.418298 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efbd953-1405-4c24-b075-51774fd521f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:43.418561 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efbd953-1405-4c24-b075-51774fd521f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:43.419212 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efbd953-1405-4c24-b075-51774fd521f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:43.421158 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efbd953-1405-4c24-b075-51774fd521f0 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:43.445119 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1263/2512] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:28:43 2022] POST /identity/v3/users => generated 291 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:28:43.451117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b65495-0f9a-4302-bd9d-369e2771e685 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:43.464384 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b65495-0f9a-4302-bd9d-369e2771e685 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:43.470583 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b34860a-6858-40d5-8c4b-313bda985b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:43.471175 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34860a-6858-40d5-8c4b-313bda985b24 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:43.471450 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34860a-6858-40d5-8c4b-313bda985b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:43.471715 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34860a-6858-40d5-8c4b-313bda985b24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:43.472357 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b34860a-6858-40d5-8c4b-313bda985b24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:43.473956 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b34860a-6858-40d5-8c4b-313bda985b24 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:43.484242 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1250/2513] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:28:43.488161 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efbd953-1405-4c24-b075-51774fd521f0 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:43.494847 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:28:43.498986 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efbd953-1405-4c24-b075-51774fd521f0 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:43.505462 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb004c-2a6d-4a29-8546-cc5fa6e8f96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:43.506100 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb004c-2a6d-4a29-8546-cc5fa6e8f96e admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:43.506606 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb004c-2a6d-4a29-8546-cc5fa6e8f96e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:43.507008 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb004c-2a6d-4a29-8546-cc5fa6e8f96e admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:43.507703 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb004c-2a6d-4a29-8546-cc5fa6e8f96e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:43.509554 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb004c-2a6d-4a29-8546-cc5fa6e8f96e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:43.520321 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1264/2514] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:28:43.524491 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b34860a-6858-40d5-8c4b-313bda985b24 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:43.534537 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b34860a-6858-40d5-8c4b-313bda985b24 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:43.539574 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faf0bdd-d379-44a6-a861-47cfb4dcc772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:43.540210 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf0bdd-d379-44a6-a861-47cfb4dcc772 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:43.540484 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf0bdd-d379-44a6-a861-47cfb4dcc772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:43.540754 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf0bdd-d379-44a6-a861-47cfb4dcc772 admin admin] PATH_INFO: `/v3/projects/8c49880996934598beb1f5710333c118/users/22d9ac50ae6e45d28515ffec737ce774/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:43.555955 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faf0bdd-d379-44a6-a861-47cfb4dcc772 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c49880996934598beb1f5710333c118, user_id=22d9ac50ae6e45d28515ffec737ce774, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:43.558650 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faf0bdd-d379-44a6-a861-47cfb4dcc772 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:43.575100 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1251/2515] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:28:43 2022] PUT /identity/v3/projects/8c49880996934598beb1f5710333c118/users/22d9ac50ae6e45d28515ffec737ce774/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:28:43.586584 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6cb92-7845-405a-8d29-eeacb75a476a None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:43.586867 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6cb92-7845-405a-8d29-eeacb75a476a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:43.587133 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6cb92-7845-405a-8d29-eeacb75a476a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:43.615532 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-c5c6cb92-7845-405a-8d29-eeacb75a476a None None] MFA Rules not processed for user `22d9ac50ae6e45d28515ffec737ce774`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:28:43.616715 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c5c6cb92-7845-405a-8d29-eeacb75a476a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:43.692062 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1265/2516] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:28:43 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:28:43.692852 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faf0bdd-d379-44a6-a861-47cfb4dcc772 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:43.764951 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faf0bdd-d379-44a6-a861-47cfb4dcc772 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:43.770466 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c0c7ae-d342-4078-9b38-f08fc797f5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:43.771071 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0c7ae-d342-4078-9b38-f08fc797f5e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:43.771350 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0c7ae-d342-4078-9b38-f08fc797f5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:43.771616 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0c7ae-d342-4078-9b38-f08fc797f5e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:43.772271 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c0c7ae-d342-4078-9b38-f08fc797f5e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:43.774233 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c0c7ae-d342-4078-9b38-f08fc797f5e3 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:43.804712 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1252/2517] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:28:43 2022] POST /identity/v3/projects => generated 381 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:28:43.809087 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6cb92-7845-405a-8d29-eeacb75a476a None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:43.820259 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6cb92-7845-405a-8d29-eeacb75a476a None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:43.878991 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b077ce-89dd-4b34-a070-1fb11d969524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:43.879613 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b077ce-89dd-4b34-a070-1fb11d969524 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:43.879891 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b077ce-89dd-4b34-a070-1fb11d969524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:43.880159 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b077ce-89dd-4b34-a070-1fb11d969524 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:43.880801 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b077ce-89dd-4b34-a070-1fb11d969524 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:43.882767 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b077ce-89dd-4b34-a070-1fb11d969524 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:43.905990 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1266/2518] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:28:43 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:28:43.910609 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c0c7ae-d342-4078-9b38-f08fc797f5e3 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:43.924522 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c0c7ae-d342-4078-9b38-f08fc797f5e3 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:43.929737 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f4bdb6-490b-4d45-af7f-59a123c5bd8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:43.930365 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f4bdb6-490b-4d45-af7f-59a123c5bd8f admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:43.930643 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f4bdb6-490b-4d45-af7f-59a123c5bd8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:43.930901 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f4bdb6-490b-4d45-af7f-59a123c5bd8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:43.931541 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f4bdb6-490b-4d45-af7f-59a123c5bd8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:43.933143 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f4bdb6-490b-4d45-af7f-59a123c5bd8f admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:43.938567 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:28:43.944616 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1253/2519] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:28:43.949106 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b077ce-89dd-4b34-a070-1fb11d969524 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:43.959931 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b077ce-89dd-4b34-a070-1fb11d969524 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:43.965717 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da99ed1-be89-4f78-8213-63de71ffd991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:43.966405 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da99ed1-be89-4f78-8213-63de71ffd991 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:43.966660 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da99ed1-be89-4f78-8213-63de71ffd991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:43.966943 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da99ed1-be89-4f78-8213-63de71ffd991 admin admin] PATH_INFO: `/v3/projects/62a1b572100c4ce68714a325353fdabb/users/c0ae4bf1e4c14cd38172596b8759f831/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:43.982307 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da99ed1-be89-4f78-8213-63de71ffd991 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62a1b572100c4ce68714a325353fdabb, user_id=c0ae4bf1e4c14cd38172596b8759f831, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:43.984974 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da99ed1-be89-4f78-8213-63de71ffd991 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:44.001435 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1267/2520] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:28:43 2022] PUT /identity/v3/projects/62a1b572100c4ce68714a325353fdabb/users/c0ae4bf1e4c14cd38172596b8759f831/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:28:44.005419 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f4bdb6-490b-4d45-af7f-59a123c5bd8f admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:44.015935 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f4bdb6-490b-4d45-af7f-59a123c5bd8f admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:44.073513 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21ebb6-f765-4c26-bfca-a70555147ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:44.074155 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21ebb6-f765-4c26-bfca-a70555147ae4 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:44.074436 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21ebb6-f765-4c26-bfca-a70555147ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:44.074707 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21ebb6-f765-4c26-bfca-a70555147ae4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:44.075352 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d21ebb6-f765-4c26-bfca-a70555147ae4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:44.076985 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d21ebb6-f765-4c26-bfca-a70555147ae4 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:44.088819 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1254/2521] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:28:44 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:28:44.093884 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da99ed1-be89-4f78-8213-63de71ffd991 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:44.108308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da99ed1-be89-4f78-8213-63de71ffd991 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:44.114289 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725930b6-43cf-4ca1-b6e4-57373c107b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:44.115004 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725930b6-43cf-4ca1-b6e4-57373c107b2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:44.115281 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725930b6-43cf-4ca1-b6e4-57373c107b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:44.115553 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725930b6-43cf-4ca1-b6e4-57373c107b2c admin admin] PATH_INFO: `/v3/projects/62a1b572100c4ce68714a325353fdabb/users/c0ae4bf1e4c14cd38172596b8759f831/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:44.117576 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725930b6-43cf-4ca1-b6e4-57373c107b2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=62a1b572100c4ce68714a325353fdabb, user_id=c0ae4bf1e4c14cd38172596b8759f831, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:44.120281 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725930b6-43cf-4ca1-b6e4-57373c107b2c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:44.131058 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1268/2522] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:28:44 2022] PUT /identity/v3/projects/62a1b572100c4ce68714a325353fdabb/users/c0ae4bf1e4c14cd38172596b8759f831/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:28:44.142984 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de336ba2-e58c-48ac-8b4b-6b600b3d568b None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:44.143238 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de336ba2-e58c-48ac-8b4b-6b600b3d568b None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:44.143506 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de336ba2-e58c-48ac-8b4b-6b600b3d568b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:44.173499 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-de336ba2-e58c-48ac-8b4b-6b600b3d568b None None] MFA Rules not processed for user `c0ae4bf1e4c14cd38172596b8759f831`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:28:44.174741 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-de336ba2-e58c-48ac-8b4b-6b600b3d568b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:44.243054 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1255/2523] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:28:44 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:28:44.254685 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725930b6-43cf-4ca1-b6e4-57373c107b2c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:44.256459 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-725930b6-43cf-4ca1-b6e4-57373c107b2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:44.299201 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:28:44.317650 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725930b6-43cf-4ca1-b6e4-57373c107b2c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:44.321499 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc93447-91f5-4e3f-8da8-403f650497e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:44.322130 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc93447-91f5-4e3f-8da8-403f650497e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:44.322403 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc93447-91f5-4e3f-8da8-403f650497e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:44.322668 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc93447-91f5-4e3f-8da8-403f650497e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:44.331955 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc93447-91f5-4e3f-8da8-403f650497e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:44.333382 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc93447-91f5-4e3f-8da8-403f650497e2 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:44.345497 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1269/2524] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:28:44 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:28:44.427374 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de336ba2-e58c-48ac-8b4b-6b600b3d568b None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:44.441494 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de336ba2-e58c-48ac-8b4b-6b600b3d568b None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:44.448064 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8203cc-9ff3-47d0-a51b-f59d84bc1c35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:44.448941 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8203cc-9ff3-47d0-a51b-f59d84bc1c35 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:44.449276 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8203cc-9ff3-47d0-a51b-f59d84bc1c35 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:44.449624 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8203cc-9ff3-47d0-a51b-f59d84bc1c35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:44.510519 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8203cc-9ff3-47d0-a51b-f59d84bc1c35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:44.511989 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8203cc-9ff3-47d0-a51b-f59d84bc1c35 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:44.521213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1256/2525] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:28:44 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:28:44.796538 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc93447-91f5-4e3f-8da8-403f650497e2 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:44.862496 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc93447-91f5-4e3f-8da8-403f650497e2 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:44.867306 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aab88c-e1cb-4084-9066-238b3c31f121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:44.867934 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aab88c-e1cb-4084-9066-238b3c31f121 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:44.868212 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aab88c-e1cb-4084-9066-238b3c31f121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:44.868480 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aab88c-e1cb-4084-9066-238b3c31f121 admin admin] PATH_INFO: `/v3/users/22d9ac50ae6e45d28515ffec737ce774` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:44.869638 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aab88c-e1cb-4084-9066-238b3c31f121 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22d9ac50ae6e45d28515ffec737ce774)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:44.871645 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aab88c-e1cb-4084-9066-238b3c31f121 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:44.918485 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-81aab88c-e1cb-4084-9066-238b3c31f121 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22d9ac50ae6e45d28515ffec737ce774'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:44.925444 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-81aab88c-e1cb-4084-9066-238b3c31f121 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22d9ac50ae6e45d28515ffec737ce774'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:44.934059 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-81aab88c-e1cb-4084-9066-238b3c31f121 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22d9ac50ae6e45d28515ffec737ce774'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:44.934685 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-81aab88c-e1cb-4084-9066-238b3c31f121 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22d9ac50ae6e45d28515ffec737ce774'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:44.946792 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-81aab88c-e1cb-4084-9066-238b3c31f121 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22d9ac50ae6e45d28515ffec737ce774'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:44.952718 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1270/2526] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:28:44 2022] DELETE /identity/v3/users/22d9ac50ae6e45d28515ffec737ce774 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:28:44.957287 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef91bb-69e3-4ab6-960a-cdf3c7e65d33 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:44.957783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef91bb-69e3-4ab6-960a-cdf3c7e65d33 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:44.957881 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef91bb-69e3-4ab6-960a-cdf3c7e65d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:44.977335 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-b5ef91bb-69e3-4ab6-960a-cdf3c7e65d33 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:28:44.978563 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b5ef91bb-69e3-4ab6-960a-cdf3c7e65d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:44.991161 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:28:45.042314 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1257/2527] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:28:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:28:45.052392 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aab88c-e1cb-4084-9066-238b3c31f121 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:45.054759 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-81aab88c-e1cb-4084-9066-238b3c31f121 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:45.117382 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aab88c-e1cb-4084-9066-238b3c31f121 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:45.121283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00833fc0-c0c9-4fc0-bdc2-0e6548aa6257 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:45.121920 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00833fc0-c0c9-4fc0-bdc2-0e6548aa6257 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:45.122198 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00833fc0-c0c9-4fc0-bdc2-0e6548aa6257 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:45.122454 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00833fc0-c0c9-4fc0-bdc2-0e6548aa6257 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:45.132506 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00833fc0-c0c9-4fc0-bdc2-0e6548aa6257 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:45.133943 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00833fc0-c0c9-4fc0-bdc2-0e6548aa6257 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:45.138866 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:28:45.144136 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1271/2528] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:28:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:28:45.425713 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ef91bb-69e3-4ab6-960a-cdf3c7e65d33 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:45.427264 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b5ef91bb-69e3-4ab6-960a-cdf3c7e65d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:45.447530 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ef91bb-69e3-4ab6-960a-cdf3c7e65d33 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:45.452866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc166b1f-3fa0-4521-afbf-168c1f0a931d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:45.453430 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc166b1f-3fa0-4521-afbf-168c1f0a931d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:45.453945 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc166b1f-3fa0-4521-afbf-168c1f0a931d admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:45.454088 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc166b1f-3fa0-4521-afbf-168c1f0a931d admin admin] PATH_INFO: `/v3/projects/8c49880996934598beb1f5710333c118` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:45.455313 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc166b1f-3fa0-4521-afbf-168c1f0a931d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c49880996934598beb1f5710333c118)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:45.457240 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc166b1f-3fa0-4521-afbf-168c1f0a931d admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:45.507237 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-fc166b1f-3fa0-4521-afbf-168c1f0a931d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c49880996934598beb1f5710333c118', 'request_id': 'req-fc166b1f-3fa0-4521-afbf-168c1f0a931d'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:45.513806 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-fc166b1f-3fa0-4521-afbf-168c1f0a931d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c49880996934598beb1f5710333c118', 'request_id': 'req-fc166b1f-3fa0-4521-afbf-168c1f0a931d'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:45.519767 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-fc166b1f-3fa0-4521-afbf-168c1f0a931d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c49880996934598beb1f5710333c118', 'request_id': 'req-fc166b1f-3fa0-4521-afbf-168c1f0a931d'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:45.533684 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-fc166b1f-3fa0-4521-afbf-168c1f0a931d admin admin] The token cache is being invalidate because project 8c49880996934598beb1f5710333c118 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:28:45.533968 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-fc166b1f-3fa0-4521-afbf-168c1f0a931d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:45.534641 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-fc166b1f-3fa0-4521-afbf-168c1f0a931d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:45.536159 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1258/2529] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:28:45 2022] DELETE /identity/v3/projects/8c49880996934598beb1f5710333c118 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:28:45.540343 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00833fc0-c0c9-4fc0-bdc2-0e6548aa6257 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:45.550499 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00833fc0-c0c9-4fc0-bdc2-0e6548aa6257 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:45.608057 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13f169a-f505-467d-b129-9131b8168d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:45.608715 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13f169a-f505-467d-b129-9131b8168d1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:45.608962 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13f169a-f505-467d-b129-9131b8168d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:45.609256 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13f169a-f505-467d-b129-9131b8168d1c admin admin] PATH_INFO: `/v3/users/c0ae4bf1e4c14cd38172596b8759f831` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:45.610463 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13f169a-f505-467d-b129-9131b8168d1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0ae4bf1e4c14cd38172596b8759f831)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:45.612389 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13f169a-f505-467d-b129-9131b8168d1c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:45.661839 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e13f169a-f505-467d-b129-9131b8168d1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0ae4bf1e4c14cd38172596b8759f831'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:45.669000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e13f169a-f505-467d-b129-9131b8168d1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0ae4bf1e4c14cd38172596b8759f831'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:45.677949 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e13f169a-f505-467d-b129-9131b8168d1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0ae4bf1e4c14cd38172596b8759f831'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:45.678589 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e13f169a-f505-467d-b129-9131b8168d1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0ae4bf1e4c14cd38172596b8759f831'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:45.690840 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e13f169a-f505-467d-b129-9131b8168d1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0ae4bf1e4c14cd38172596b8759f831'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:45.696941 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1272/2530] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:28:45 2022] DELETE /identity/v3/users/c0ae4bf1e4c14cd38172596b8759f831 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:28:45.986886 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc166b1f-3fa0-4521-afbf-168c1f0a931d admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:45.988492 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-fc166b1f-3fa0-4521-afbf-168c1f0a931d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:46.001614 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc166b1f-3fa0-4521-afbf-168c1f0a931d admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:46.061163 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ae123b-d222-4589-947d-78326bd79656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:46.062281 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae123b-d222-4589-947d-78326bd79656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:46.062777 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae123b-d222-4589-947d-78326bd79656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:46.063267 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae123b-d222-4589-947d-78326bd79656 admin admin] PATH_INFO: `/v3/projects/62a1b572100c4ce68714a325353fdabb` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:46.064920 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ae123b-d222-4589-947d-78326bd79656 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62a1b572100c4ce68714a325353fdabb)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:46.066884 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ae123b-d222-4589-947d-78326bd79656 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:46.115438 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:28:46.118393 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-e9ae123b-d222-4589-947d-78326bd79656 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62a1b572100c4ce68714a325353fdabb', 'request_id': 'req-e9ae123b-d222-4589-947d-78326bd79656'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:46.125667 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-e9ae123b-d222-4589-947d-78326bd79656 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62a1b572100c4ce68714a325353fdabb', 'request_id': 'req-e9ae123b-d222-4589-947d-78326bd79656'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:46.131465 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-e9ae123b-d222-4589-947d-78326bd79656 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62a1b572100c4ce68714a325353fdabb', 'request_id': 'req-e9ae123b-d222-4589-947d-78326bd79656'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:46.144922 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-e9ae123b-d222-4589-947d-78326bd79656 admin admin] The token cache is being invalidate because project 62a1b572100c4ce68714a325353fdabb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:28:46.145202 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-e9ae123b-d222-4589-947d-78326bd79656 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:46.145809 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-e9ae123b-d222-4589-947d-78326bd79656 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:46.147398 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1259/2531] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:28:45 2022] DELETE /identity/v3/projects/62a1b572100c4ce68714a325353fdabb => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:28:46.152442 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886265f1-c2fa-4962-a98d-2dd56cf8d2d0 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:46.152911 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886265f1-c2fa-4962-a98d-2dd56cf8d2d0 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:46.153025 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886265f1-c2fa-4962-a98d-2dd56cf8d2d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:46.173914 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-886265f1-c2fa-4962-a98d-2dd56cf8d2d0 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:28:46.175145 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-886265f1-c2fa-4962-a98d-2dd56cf8d2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:46.238652 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1273/2532] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:28:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:28:46.249825 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e46074-d3ff-457b-97d1-91b7c202b97e None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:46.250146 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e46074-d3ff-457b-97d1-91b7c202b97e None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:46.250415 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e46074-d3ff-457b-97d1-91b7c202b97e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:46.271538 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-c1e46074-d3ff-457b-97d1-91b7c202b97e None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:28:46.272739 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c1e46074-d3ff-457b-97d1-91b7c202b97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:46.282110 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1260/2533] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:28:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:28:46.286648 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886265f1-c2fa-4962-a98d-2dd56cf8d2d0 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:46.297397 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886265f1-c2fa-4962-a98d-2dd56cf8d2d0 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:46.302469 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f117c2-56d1-4e50-aee9-dcb590805849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:46.303069 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f117c2-56d1-4e50-aee9-dcb590805849 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:46.303346 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f117c2-56d1-4e50-aee9-dcb590805849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:46.303614 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f117c2-56d1-4e50-aee9-dcb590805849 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:46.304261 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f117c2-56d1-4e50-aee9-dcb590805849 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:28:46.304563 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f117c2-56d1-4e50-aee9-dcb590805849 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:46.306214 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f117c2-56d1-4e50-aee9-dcb590805849 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:46.319014 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1274/2534] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:28:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:28:46.323315 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e46074-d3ff-457b-97d1-91b7c202b97e None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:46.332950 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e46074-d3ff-457b-97d1-91b7c202b97e None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:46.339113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69eb032-f2ef-4258-9866-8b0d33659281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:46.339715 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69eb032-f2ef-4258-9866-8b0d33659281 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:46.339993 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69eb032-f2ef-4258-9866-8b0d33659281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:46.340259 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69eb032-f2ef-4258-9866-8b0d33659281 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:46.340895 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69eb032-f2ef-4258-9866-8b0d33659281 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:46.342926 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69eb032-f2ef-4258-9866-8b0d33659281 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:46.374711 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1261/2535] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:28:46 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:28:46.379065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f117c2-56d1-4e50-aee9-dcb590805849 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:46.389531 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f117c2-56d1-4e50-aee9-dcb590805849 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:46.393348 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:28:46.451247 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2776c-20c6-4be4-8093-f21b2c228ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:46.451858 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2776c-20c6-4be4-8093-f21b2c228ebf admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:46.452135 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2776c-20c6-4be4-8093-f21b2c228ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:46.452401 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2776c-20c6-4be4-8093-f21b2c228ebf admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:46.453031 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df2776c-20c6-4be4-8093-f21b2c228ebf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:46.454980 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df2776c-20c6-4be4-8093-f21b2c228ebf admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:46.478622 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1275/2536] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:28:46 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:28:46.482799 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69eb032-f2ef-4258-9866-8b0d33659281 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:46.493156 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69eb032-f2ef-4258-9866-8b0d33659281 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:46.497973 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf2c748-0dea-4cf4-8dfe-6854257652ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:46.498679 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2c748-0dea-4cf4-8dfe-6854257652ae admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:46.498951 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2c748-0dea-4cf4-8dfe-6854257652ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:46.499213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2c748-0dea-4cf4-8dfe-6854257652ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:46.499848 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf2c748-0dea-4cf4-8dfe-6854257652ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:46.501442 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf2c748-0dea-4cf4-8dfe-6854257652ae admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:46.511866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1262/2537] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:28:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 05:28:46.515997 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2776c-20c6-4be4-8093-f21b2c228ebf admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:46.526859 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2776c-20c6-4be4-8093-f21b2c228ebf admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:46.532145 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a24346-f7ab-445d-83b6-84d02280a583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:46.532752 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a24346-f7ab-445d-83b6-84d02280a583 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:46.533049 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a24346-f7ab-445d-83b6-84d02280a583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:46.533339 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a24346-f7ab-445d-83b6-84d02280a583 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:46.534017 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a24346-f7ab-445d-83b6-84d02280a583 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:46.535622 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a24346-f7ab-445d-83b6-84d02280a583 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:46.546118 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1276/2538] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:28:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:28:46.550356 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf2c748-0dea-4cf4-8dfe-6854257652ae admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:46.560770 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf2c748-0dea-4cf4-8dfe-6854257652ae admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:46.566415 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf956c-1834-431b-9487-3f2a6f970ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:46.567070 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf956c-1834-431b-9487-3f2a6f970ea1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:46.567346 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf956c-1834-431b-9487-3f2a6f970ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:46.567614 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf956c-1834-431b-9487-3f2a6f970ea1 admin admin] PATH_INFO: `/v3/projects/2ba9f669c9504aee8e7e0a4cdaa17f64/users/f363692dfed94e6da4cf11eba22e41f7/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:46.582682 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bf956c-1834-431b-9487-3f2a6f970ea1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ba9f669c9504aee8e7e0a4cdaa17f64, user_id=f363692dfed94e6da4cf11eba22e41f7, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:46.585378 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bf956c-1834-431b-9487-3f2a6f970ea1 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:46.601149 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1263/2539] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:28:46 2022] PUT /identity/v3/projects/2ba9f669c9504aee8e7e0a4cdaa17f64/users/f363692dfed94e6da4cf11eba22e41f7/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:28:46.611589 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c5306d-5b51-4635-9fe9-4987f10e426f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:46.611917 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c5306d-5b51-4635-9fe9-4987f10e426f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:46.612180 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c5306d-5b51-4635-9fe9-4987f10e426f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:46.640854 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e6c5306d-5b51-4635-9fe9-4987f10e426f None None] MFA Rules not processed for user `f363692dfed94e6da4cf11eba22e41f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:28:46.642149 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e6c5306d-5b51-4635-9fe9-4987f10e426f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:46.710140 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1277/2540] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:28:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:28:46.714712 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf956c-1834-431b-9487-3f2a6f970ea1 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:46.776877 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf956c-1834-431b-9487-3f2a6f970ea1 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:46.782470 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001e6316-516c-4c2a-ac2f-aff963bc21f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:46.783071 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e6316-516c-4c2a-ac2f-aff963bc21f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:46.783372 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e6316-516c-4c2a-ac2f-aff963bc21f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:46.783676 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e6316-516c-4c2a-ac2f-aff963bc21f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:46.784404 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001e6316-516c-4c2a-ac2f-aff963bc21f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:46.786513 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001e6316-516c-4c2a-ac2f-aff963bc21f9 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:46.815925 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1264/2541] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:28:46 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:28:46.820087 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c5306d-5b51-4635-9fe9-4987f10e426f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:46.832039 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c5306d-5b51-4635-9fe9-4987f10e426f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:46.889929 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf8eec-02e5-43b9-b41f-a18c40b7b653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:46.890563 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf8eec-02e5-43b9-b41f-a18c40b7b653 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:46.890837 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf8eec-02e5-43b9-b41f-a18c40b7b653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:46.891103 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf8eec-02e5-43b9-b41f-a18c40b7b653 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:46.891753 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaf8eec-02e5-43b9-b41f-a18c40b7b653 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:46.893691 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaf8eec-02e5-43b9-b41f-a18c40b7b653 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:46.916587 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1278/2542] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:28:46 2022] POST /identity/v3/users => generated 298 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:28:46.921238 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001e6316-516c-4c2a-ac2f-aff963bc21f9 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:46.932250 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001e6316-516c-4c2a-ac2f-aff963bc21f9 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:46.937797 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d3933-5e51-499a-9179-4283b5c0f130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:46.938440 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d3933-5e51-499a-9179-4283b5c0f130 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:46.938711 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d3933-5e51-499a-9179-4283b5c0f130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:46.938975 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d3933-5e51-499a-9179-4283b5c0f130 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:46.939624 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51d3933-5e51-499a-9179-4283b5c0f130 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:46.941230 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51d3933-5e51-499a-9179-4283b5c0f130 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:46.952318 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1265/2543] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:28:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:28:46.956997 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf8eec-02e5-43b9-b41f-a18c40b7b653 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:46.965620 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:28:46.970641 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf8eec-02e5-43b9-b41f-a18c40b7b653 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:46.976674 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d889c1-1c62-499a-8bbc-c18b018f6ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:46.977324 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d889c1-1c62-499a-8bbc-c18b018f6ea7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:46.977611 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d889c1-1c62-499a-8bbc-c18b018f6ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:46.977874 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d889c1-1c62-499a-8bbc-c18b018f6ea7 admin admin] PATH_INFO: `/v3/projects/bc3f76db672f41ea988b3596a1585e2e/users/e30f8599a4064313aa61203f99ab4373/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:46.993871 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d889c1-1c62-499a-8bbc-c18b018f6ea7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc3f76db672f41ea988b3596a1585e2e, user_id=e30f8599a4064313aa61203f99ab4373, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:46.996672 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d889c1-1c62-499a-8bbc-c18b018f6ea7 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:47.012918 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1279/2544] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:28:46 2022] PUT /identity/v3/projects/bc3f76db672f41ea988b3596a1585e2e/users/e30f8599a4064313aa61203f99ab4373/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:28:47.017012 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d3933-5e51-499a-9179-4283b5c0f130 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:47.027458 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d3933-5e51-499a-9179-4283b5c0f130 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:47.070363 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:28:47.092099 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba8358-d8cc-49e3-90a7-39f24e7b7594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:47.092735 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba8358-d8cc-49e3-90a7-39f24e7b7594 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:47.093007 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba8358-d8cc-49e3-90a7-39f24e7b7594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:47.093261 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba8358-d8cc-49e3-90a7-39f24e7b7594 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:47.093896 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba8358-d8cc-49e3-90a7-39f24e7b7594 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:47.095835 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba8358-d8cc-49e3-90a7-39f24e7b7594 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:47.106756 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1266/2545] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:28:47.111158 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d889c1-1c62-499a-8bbc-c18b018f6ea7 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:47.121551 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d889c1-1c62-499a-8bbc-c18b018f6ea7 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:47.126368 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c5013-1202-40b9-aab3-f76c28c714c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:47.127009 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c5013-1202-40b9-aab3-f76c28c714c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:47.127283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c5013-1202-40b9-aab3-f76c28c714c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:47.127530 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c5013-1202-40b9-aab3-f76c28c714c6 admin admin] PATH_INFO: `/v3/projects/bc3f76db672f41ea988b3596a1585e2e/users/e30f8599a4064313aa61203f99ab4373/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:47.129486 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8c5013-1202-40b9-aab3-f76c28c714c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc3f76db672f41ea988b3596a1585e2e, user_id=e30f8599a4064313aa61203f99ab4373, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:47.132163 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8c5013-1202-40b9-aab3-f76c28c714c6 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:47.142477 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1280/2546] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:28:47 2022] PUT /identity/v3/projects/bc3f76db672f41ea988b3596a1585e2e/users/e30f8599a4064313aa61203f99ab4373/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:28:47.153474 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0573d0e-0251-45a7-8bb2-d4f569cd6c95 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:47.153751 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0573d0e-0251-45a7-8bb2-d4f569cd6c95 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:47.154041 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0573d0e-0251-45a7-8bb2-d4f569cd6c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:47.184641 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-a0573d0e-0251-45a7-8bb2-d4f569cd6c95 None None] MFA Rules not processed for user `e30f8599a4064313aa61203f99ab4373`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:28:47.185948 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a0573d0e-0251-45a7-8bb2-d4f569cd6c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:47.269153 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1267/2547] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:28:47 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:28:47.280905 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c5013-1202-40b9-aab3-f76c28c714c6 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:47.283784 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4e8c5013-1202-40b9-aab3-f76c28c714c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:47.359931 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c5013-1202-40b9-aab3-f76c28c714c6 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:47.364404 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb010024-4512-4589-8e94-a5821f3b9ecd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:47.364972 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb010024-4512-4589-8e94-a5821f3b9ecd service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:47.365271 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb010024-4512-4589-8e94-a5821f3b9ecd service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:47.365527 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb010024-4512-4589-8e94-a5821f3b9ecd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:47.424894 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb010024-4512-4589-8e94-a5821f3b9ecd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:47.426387 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb010024-4512-4589-8e94-a5821f3b9ecd service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:47.436060 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1281/2548] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:28:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:28:50.073021 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0573d0e-0251-45a7-8bb2-d4f569cd6c95 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:50.083839 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0573d0e-0251-45a7-8bb2-d4f569cd6c95 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:50.087858 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4ec8d2-d9da-4d16-84e7-826e0dd25ff2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:50.088469 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ec8d2-d9da-4d16-84e7-826e0dd25ff2 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:50.088743 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ec8d2-d9da-4d16-84e7-826e0dd25ff2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:50.089010 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ec8d2-d9da-4d16-84e7-826e0dd25ff2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:50.091002 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-cc4ec8d2-d9da-4d16-84e7-826e0dd25ff2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:50.165546 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4ec8d2-d9da-4d16-84e7-826e0dd25ff2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:50.167036 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4ec8d2-d9da-4d16-84e7-826e0dd25ff2 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:50.178303 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1268/2549] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:28:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:28:51.753637 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb010024-4512-4589-8e94-a5821f3b9ecd service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:51.755551 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-eb010024-4512-4589-8e94-a5821f3b9ecd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:51.768375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb010024-4512-4589-8e94-a5821f3b9ecd service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:51.772638 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97291f6-e317-4599-9559-6ac5ca69b603 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:51.773251 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97291f6-e317-4599-9559-6ac5ca69b603 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:51.773526 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97291f6-e317-4599-9559-6ac5ca69b603 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:51.773790 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97291f6-e317-4599-9559-6ac5ca69b603 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:51.783335 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97291f6-e317-4599-9559-6ac5ca69b603 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:51.784769 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97291f6-e317-4599-9559-6ac5ca69b603 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:51.794298 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1282/2550] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:28:51 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:28:56.770279 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4ec8d2-d9da-4d16-84e7-826e0dd25ff2 service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:56.827934 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:28:56.841200 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4ec8d2-d9da-4d16-84e7-826e0dd25ff2 service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:56.846579 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff815a-5f92-4361-976d-6c55edebd613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:56.847212 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff815a-5f92-4361-976d-6c55edebd613 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:56.847469 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff815a-5f92-4361-976d-6c55edebd613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:56.847733 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff815a-5f92-4361-976d-6c55edebd613 admin admin] PATH_INFO: `/v3/users/f363692dfed94e6da4cf11eba22e41f7` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:56.848867 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff815a-5f92-4361-976d-6c55edebd613 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f363692dfed94e6da4cf11eba22e41f7)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:56.850850 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff815a-5f92-4361-976d-6c55edebd613 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:56.898993 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c0ff815a-5f92-4361-976d-6c55edebd613 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f363692dfed94e6da4cf11eba22e41f7'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:56.906296 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c0ff815a-5f92-4361-976d-6c55edebd613 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f363692dfed94e6da4cf11eba22e41f7'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:56.907090 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c0ff815a-5f92-4361-976d-6c55edebd613 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f363692dfed94e6da4cf11eba22e41f7'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:56.916367 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c0ff815a-5f92-4361-976d-6c55edebd613 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f363692dfed94e6da4cf11eba22e41f7'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:56.916998 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c0ff815a-5f92-4361-976d-6c55edebd613 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f363692dfed94e6da4cf11eba22e41f7'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:56.934371 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1269/2551] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:28:56 2022] DELETE /identity/v3/users/f363692dfed94e6da4cf11eba22e41f7 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:28:56.939122 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad1b53-bf38-48bf-9ae3-da115e98845b None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:56.939614 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad1b53-bf38-48bf-9ae3-da115e98845b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:56.939713 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad1b53-bf38-48bf-9ae3-da115e98845b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:56.959707 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-86ad1b53-bf38-48bf-9ae3-da115e98845b None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:28:56.960919 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-86ad1b53-bf38-48bf-9ae3-da115e98845b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:57.023027 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1283/2552] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:28:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:28:57.033192 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff815a-5f92-4361-976d-6c55edebd613 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:57.034702 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c0ff815a-5f92-4361-976d-6c55edebd613 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:57.094598 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff815a-5f92-4361-976d-6c55edebd613 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:57.098581 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6189b68-2d77-41ed-b4e7-0e8def33e05d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:57.099177 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6189b68-2d77-41ed-b4e7-0e8def33e05d service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:57.099449 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6189b68-2d77-41ed-b4e7-0e8def33e05d service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:57.099713 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6189b68-2d77-41ed-b4e7-0e8def33e05d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:57.109968 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6189b68-2d77-41ed-b4e7-0e8def33e05d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:57.111544 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6189b68-2d77-41ed-b4e7-0e8def33e05d service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:57.121112 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1270/2553] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:28:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:28:57.401076 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ad1b53-bf38-48bf-9ae3-da115e98845b None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:57.402700 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-86ad1b53-bf38-48bf-9ae3-da115e98845b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:57.418277 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ad1b53-bf38-48bf-9ae3-da115e98845b None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:57.424299 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd07215-1808-498f-9e7c-2eb24e9ad7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:57.424906 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd07215-1808-498f-9e7c-2eb24e9ad7a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:57.425181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd07215-1808-498f-9e7c-2eb24e9ad7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:57.425818 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd07215-1808-498f-9e7c-2eb24e9ad7a9 admin admin] PATH_INFO: `/v3/projects/2ba9f669c9504aee8e7e0a4cdaa17f64` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:57.427063 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd07215-1808-498f-9e7c-2eb24e9ad7a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ba9f669c9504aee8e7e0a4cdaa17f64)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:57.429159 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd07215-1808-498f-9e7c-2eb24e9ad7a9 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:57.432249 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:28:57.485415 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6dd07215-1808-498f-9e7c-2eb24e9ad7a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ba9f669c9504aee8e7e0a4cdaa17f64', 'request_id': 'req-6dd07215-1808-498f-9e7c-2eb24e9ad7a9'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:57.491717 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6dd07215-1808-498f-9e7c-2eb24e9ad7a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ba9f669c9504aee8e7e0a4cdaa17f64', 'request_id': 'req-6dd07215-1808-498f-9e7c-2eb24e9ad7a9'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:57.500220 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6dd07215-1808-498f-9e7c-2eb24e9ad7a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ba9f669c9504aee8e7e0a4cdaa17f64', 'request_id': 'req-6dd07215-1808-498f-9e7c-2eb24e9ad7a9'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:57.511307 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6dd07215-1808-498f-9e7c-2eb24e9ad7a9 admin admin] The token cache is being invalidate because project 2ba9f669c9504aee8e7e0a4cdaa17f64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:28:57.511596 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6dd07215-1808-498f-9e7c-2eb24e9ad7a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:57.512221 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6dd07215-1808-498f-9e7c-2eb24e9ad7a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:57.513767 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1284/2554] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:28:57 2022] DELETE /identity/v3/projects/2ba9f669c9504aee8e7e0a4cdaa17f64 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:28:57.518025 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6189b68-2d77-41ed-b4e7-0e8def33e05d service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:57.527688 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6189b68-2d77-41ed-b4e7-0e8def33e05d service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:57.588661 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d81989-f0c8-4849-82db-eb46d1cb5e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:57.589304 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d81989-f0c8-4849-82db-eb46d1cb5e33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:57.589583 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d81989-f0c8-4849-82db-eb46d1cb5e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:57.589858 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d81989-f0c8-4849-82db-eb46d1cb5e33 admin admin] PATH_INFO: `/v3/users/e30f8599a4064313aa61203f99ab4373` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:57.591032 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d81989-f0c8-4849-82db-eb46d1cb5e33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e30f8599a4064313aa61203f99ab4373)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:57.593018 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d81989-f0c8-4849-82db-eb46d1cb5e33 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:57.607280 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:28:57.642386 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-d3d81989-f0c8-4849-82db-eb46d1cb5e33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e30f8599a4064313aa61203f99ab4373'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:57.649088 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-d3d81989-f0c8-4849-82db-eb46d1cb5e33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e30f8599a4064313aa61203f99ab4373'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:57.649777 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-d3d81989-f0c8-4849-82db-eb46d1cb5e33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e30f8599a4064313aa61203f99ab4373'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:57.658705 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-d3d81989-f0c8-4849-82db-eb46d1cb5e33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e30f8599a4064313aa61203f99ab4373'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:57.659386 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-d3d81989-f0c8-4849-82db-eb46d1cb5e33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e30f8599a4064313aa61203f99ab4373'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:57.678244 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1271/2555] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:28:57 2022] DELETE /identity/v3/users/e30f8599a4064313aa61203f99ab4373 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:28:57.975165 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd07215-1808-498f-9e7c-2eb24e9ad7a9 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:57.976735 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6dd07215-1808-498f-9e7c-2eb24e9ad7a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:58.001200 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd07215-1808-498f-9e7c-2eb24e9ad7a9 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:58.062321 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feee5d49-d733-4e53-b9fd-b2ed70085b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:58.062972 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee5d49-d733-4e53-b9fd-b2ed70085b46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:58.063426 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee5d49-d733-4e53-b9fd-b2ed70085b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:58.063548 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee5d49-d733-4e53-b9fd-b2ed70085b46 admin admin] PATH_INFO: `/v3/projects/bc3f76db672f41ea988b3596a1585e2e` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:58.064658 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feee5d49-d733-4e53-b9fd-b2ed70085b46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc3f76db672f41ea988b3596a1585e2e)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:58.066612 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feee5d49-d733-4e53-b9fd-b2ed70085b46 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:58.118083 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-feee5d49-d733-4e53-b9fd-b2ed70085b46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc3f76db672f41ea988b3596a1585e2e', 'request_id': 'req-feee5d49-d733-4e53-b9fd-b2ed70085b46'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:58.124705 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-feee5d49-d733-4e53-b9fd-b2ed70085b46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc3f76db672f41ea988b3596a1585e2e', 'request_id': 'req-feee5d49-d733-4e53-b9fd-b2ed70085b46'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:58.132346 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-feee5d49-d733-4e53-b9fd-b2ed70085b46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc3f76db672f41ea988b3596a1585e2e', 'request_id': 'req-feee5d49-d733-4e53-b9fd-b2ed70085b46'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:58.143483 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-feee5d49-d733-4e53-b9fd-b2ed70085b46 admin admin] The token cache is being invalidate because project bc3f76db672f41ea988b3596a1585e2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:28:58.143766 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-feee5d49-d733-4e53-b9fd-b2ed70085b46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:58.144400 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-feee5d49-d733-4e53-b9fd-b2ed70085b46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:28:58.146258 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1285/2556] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:28:57 2022] DELETE /identity/v3/projects/bc3f76db672f41ea988b3596a1585e2e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:28:58.152691 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b815a46-2628-425a-8a23-3b3922807578 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:58.153216 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b815a46-2628-425a-8a23-3b3922807578 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:58.153216 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b815a46-2628-425a-8a23-3b3922807578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:58.174876 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-8b815a46-2628-425a-8a23-3b3922807578 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:28:58.176415 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-8b815a46-2628-425a-8a23-3b3922807578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:58.242507 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1272/2557] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:28:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:28:58.251001 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc80495-cc2e-40c1-92e3-a17a1b741944 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:58.251360 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc80495-cc2e-40c1-92e3-a17a1b741944 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:58.251716 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc80495-cc2e-40c1-92e3-a17a1b741944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:58.272299 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-7cc80495-cc2e-40c1-92e3-a17a1b741944 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:28:58.273508 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7cc80495-cc2e-40c1-92e3-a17a1b741944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:58.283502 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1286/2558] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:28:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:28:58.287781 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b815a46-2628-425a-8a23-3b3922807578 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:58.298351 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b815a46-2628-425a-8a23-3b3922807578 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:58.303223 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3828589-4015-4478-948c-f85d17400d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:58.303866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3828589-4015-4478-948c-f85d17400d87 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:58.304144 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3828589-4015-4478-948c-f85d17400d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:58.304406 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3828589-4015-4478-948c-f85d17400d87 admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:58.305065 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3828589-4015-4478-948c-f85d17400d87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:28:58.305366 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3828589-4015-4478-948c-f85d17400d87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:58.307085 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3828589-4015-4478-948c-f85d17400d87 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:58.319698 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1273/2559] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:28:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:28:58.324458 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc80495-cc2e-40c1-92e3-a17a1b741944 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:58.338080 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc80495-cc2e-40c1-92e3-a17a1b741944 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:58.343074 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c57bd2-c4a2-439f-a676-f7168f448588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:58.343704 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c57bd2-c4a2-439f-a676-f7168f448588 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:58.344333 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c57bd2-c4a2-439f-a676-f7168f448588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:58.344709 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c57bd2-c4a2-439f-a676-f7168f448588 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:58.345348 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c57bd2-c4a2-439f-a676-f7168f448588 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:58.347322 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c57bd2-c4a2-439f-a676-f7168f448588 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:58.364641 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:28:58.380908 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1287/2560] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:28:58 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:28:58.384924 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3828589-4015-4478-948c-f85d17400d87 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:58.395666 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3828589-4015-4478-948c-f85d17400d87 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:58.457415 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb133abc-ebe6-4e5e-88b4-1cd3a350c451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:58.458282 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb133abc-ebe6-4e5e-88b4-1cd3a350c451 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:58.458622 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb133abc-ebe6-4e5e-88b4-1cd3a350c451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:58.458902 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb133abc-ebe6-4e5e-88b4-1cd3a350c451 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:58.459762 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb133abc-ebe6-4e5e-88b4-1cd3a350c451 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:58.462363 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb133abc-ebe6-4e5e-88b4-1cd3a350c451 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:58.486044 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1274/2561] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:28:58 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:28:58.489993 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c57bd2-c4a2-439f-a676-f7168f448588 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:58.501149 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c57bd2-c4a2-439f-a676-f7168f448588 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:58.505959 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df636df-e15b-4833-9d6e-ca36c7c23b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:58.506590 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df636df-e15b-4833-9d6e-ca36c7c23b02 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:58.506861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df636df-e15b-4833-9d6e-ca36c7c23b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:58.507154 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df636df-e15b-4833-9d6e-ca36c7c23b02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:58.507820 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df636df-e15b-4833-9d6e-ca36c7c23b02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:58.509425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df636df-e15b-4833-9d6e-ca36c7c23b02 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:58.519898 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1288/2562] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 05:28:58.524218 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb133abc-ebe6-4e5e-88b4-1cd3a350c451 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:58.535150 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb133abc-ebe6-4e5e-88b4-1cd3a350c451 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:58.540477 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f4929-1dfc-4665-9bde-c97f948d264c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:58.541084 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f4929-1dfc-4665-9bde-c97f948d264c admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:58.541499 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f4929-1dfc-4665-9bde-c97f948d264c admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:58.541774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f4929-1dfc-4665-9bde-c97f948d264c admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:58.542512 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07f4929-1dfc-4665-9bde-c97f948d264c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:58.544151 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07f4929-1dfc-4665-9bde-c97f948d264c admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:58.546462 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:28:58.557540 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1275/2563] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:28:58.561919 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df636df-e15b-4833-9d6e-ca36c7c23b02 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:58.573358 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df636df-e15b-4833-9d6e-ca36c7c23b02 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:58.579672 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4bae15-c011-4094-b3f4-58f8cf1f6981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:58.580363 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4bae15-c011-4094-b3f4-58f8cf1f6981 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:58.580647 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4bae15-c011-4094-b3f4-58f8cf1f6981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:58.580917 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4bae15-c011-4094-b3f4-58f8cf1f6981 admin admin] PATH_INFO: `/v3/projects/4a8f7f9c72f54b9ebd2f93bde3d6d8dc/users/fb6a643b04624511b4b7fa5d0cea152f/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:58.597736 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4bae15-c011-4094-b3f4-58f8cf1f6981 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a8f7f9c72f54b9ebd2f93bde3d6d8dc, user_id=fb6a643b04624511b4b7fa5d0cea152f, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:58.600589 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4bae15-c011-4094-b3f4-58f8cf1f6981 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:58.615668 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1289/2564] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:28:58 2022] PUT /identity/v3/projects/4a8f7f9c72f54b9ebd2f93bde3d6d8dc/users/fb6a643b04624511b4b7fa5d0cea152f/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:28:58.628655 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a49f8-42bc-4604-b33f-ae21286eef30 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:58.628908 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a49f8-42bc-4604-b33f-ae21286eef30 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:58.629181 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a49f8-42bc-4604-b33f-ae21286eef30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:58.658401 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-6b4a49f8-42bc-4604-b33f-ae21286eef30 None None] MFA Rules not processed for user `fb6a643b04624511b4b7fa5d0cea152f`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:28:58.659612 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-6b4a49f8-42bc-4604-b33f-ae21286eef30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:58.727398 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1276/2565] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:28:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:28:58.732085 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4bae15-c011-4094-b3f4-58f8cf1f6981 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:58.795451 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4bae15-c011-4094-b3f4-58f8cf1f6981 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:58.800959 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b6ba4-1340-4142-a816-b091e3aa855c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:58.801721 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b6ba4-1340-4142-a816-b091e3aa855c admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:58.802129 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b6ba4-1340-4142-a816-b091e3aa855c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:58.802382 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b6ba4-1340-4142-a816-b091e3aa855c admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:58.803035 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8b6ba4-1340-4142-a816-b091e3aa855c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:58.804973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8b6ba4-1340-4142-a816-b091e3aa855c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:58.834064 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1290/2566] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:28:58 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:28:58.838961 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4a49f8-42bc-4604-b33f-ae21286eef30 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:58.851864 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4a49f8-42bc-4604-b33f-ae21286eef30 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:58.911750 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b0a3be-9d7c-447a-abda-1be890d67625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:58.912326 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b0a3be-9d7c-447a-abda-1be890d67625 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:58.912682 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b0a3be-9d7c-447a-abda-1be890d67625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:58.913003 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b0a3be-9d7c-447a-abda-1be890d67625 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:58.913723 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b0a3be-9d7c-447a-abda-1be890d67625 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:58.915884 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b0a3be-9d7c-447a-abda-1be890d67625 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:58.944895 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1277/2567] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:28:58 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:28:58.946176 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b6ba4-1340-4142-a816-b091e3aa855c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:58.957041 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b6ba4-1340-4142-a816-b091e3aa855c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:58.962222 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76663f9-30a0-44d3-ba65-2d3e921b9d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:58.962834 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76663f9-30a0-44d3-ba65-2d3e921b9d66 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:58.963106 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76663f9-30a0-44d3-ba65-2d3e921b9d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:58.963462 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76663f9-30a0-44d3-ba65-2d3e921b9d66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:58.964125 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76663f9-30a0-44d3-ba65-2d3e921b9d66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:58.965852 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76663f9-30a0-44d3-ba65-2d3e921b9d66 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:58.977291 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1291/2568] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:28:58.981452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b0a3be-9d7c-447a-abda-1be890d67625 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:58.991913 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b0a3be-9d7c-447a-abda-1be890d67625 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:58.996620 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3c002-b99c-40a6-b8c8-bf1d38e88712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:58.997226 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3c002-b99c-40a6-b8c8-bf1d38e88712 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:58.997497 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3c002-b99c-40a6-b8c8-bf1d38e88712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:58.997767 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3c002-b99c-40a6-b8c8-bf1d38e88712 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:58.998471 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f3c002-b99c-40a6-b8c8-bf1d38e88712 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:59.000095 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f3c002-b99c-40a6-b8c8-bf1d38e88712 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:59.010275 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1278/2569] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:28:59.015077 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76663f9-30a0-44d3-ba65-2d3e921b9d66 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:59.025056 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76663f9-30a0-44d3-ba65-2d3e921b9d66 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:59.029837 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5f7b77-b87e-4005-9d58-57643537026b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:59.030546 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f7b77-b87e-4005-9d58-57643537026b admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:59.030824 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f7b77-b87e-4005-9d58-57643537026b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:59.031105 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f7b77-b87e-4005-9d58-57643537026b admin admin] PATH_INFO: `/v3/projects/5559d3d0435d42339fb208420aba40a0/users/37f1f5efa55c4d488a16b9832cf2e0ef/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:59.046290 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5f7b77-b87e-4005-9d58-57643537026b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5559d3d0435d42339fb208420aba40a0, user_id=37f1f5efa55c4d488a16b9832cf2e0ef, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:59.048918 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5f7b77-b87e-4005-9d58-57643537026b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:59.063341 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1292/2570] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:28:59 2022] PUT /identity/v3/projects/5559d3d0435d42339fb208420aba40a0/users/37f1f5efa55c4d488a16b9832cf2e0ef/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:28:59.073725 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd3414-bcdd-4b8c-8504-f984c430095e None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:59.074146 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd3414-bcdd-4b8c-8504-f984c430095e None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:59.074536 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd3414-bcdd-4b8c-8504-f984c430095e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:59.107270 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-3ffd3414-bcdd-4b8c-8504-f984c430095e None None] MFA Rules not processed for user `37f1f5efa55c4d488a16b9832cf2e0ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:28:59.108591 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3ffd3414-bcdd-4b8c-8504-f984c430095e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:59.169413 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:28:59.179900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1279/2571] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:28:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:28:59.184296 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5f7b77-b87e-4005-9d58-57643537026b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:59.249635 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5f7b77-b87e-4005-9d58-57643537026b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:59.254984 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1027e0d8-e376-465c-96ba-e0d53d32e205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:59.255610 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1027e0d8-e376-465c-96ba-e0d53d32e205 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:59.255896 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1027e0d8-e376-465c-96ba-e0d53d32e205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:59.256149 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1027e0d8-e376-465c-96ba-e0d53d32e205 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:59.256784 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1027e0d8-e376-465c-96ba-e0d53d32e205 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:59.258743 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1027e0d8-e376-465c-96ba-e0d53d32e205 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:59.281147 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:28:59.282420 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ff535-b832-4f6c-9abd-6d91facfd1f1 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:59.282700 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ff535-b832-4f6c-9abd-6d91facfd1f1 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:59.282973 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ff535-b832-4f6c-9abd-6d91facfd1f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:59.293164 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1293/2572] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:28:59 2022] POST /identity/v3/projects => generated 381 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:28:59.297360 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1027e0d8-e376-465c-96ba-e0d53d32e205 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:59.303911 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-819ff535-b832-4f6c-9abd-6d91facfd1f1 None None] MFA Rules not processed for user `b0f728c7e4074084be1a433ae14879b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:28:59.305119 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-819ff535-b832-4f6c-9abd-6d91facfd1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:59.362471 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1027e0d8-e376-465c-96ba-e0d53d32e205 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:59.366583 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1280/2573] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:28:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:28:59.367618 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6be77d3-ff96-494f-936e-bd51f27fcc2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:59.368223 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be77d3-ff96-494f-936e-bd51f27fcc2b admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:59.368501 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be77d3-ff96-494f-936e-bd51f27fcc2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:59.368771 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be77d3-ff96-494f-936e-bd51f27fcc2b admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:59.369410 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6be77d3-ff96-494f-936e-bd51f27fcc2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:59.371350 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6be77d3-ff96-494f-936e-bd51f27fcc2b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:59.394051 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1294/2574] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:28:59 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:28:59.397493 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ff535-b832-4f6c-9abd-6d91facfd1f1 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:59.408804 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ff535-b832-4f6c-9abd-6d91facfd1f1 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:59.413373 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fa5f05-00d1-43f3-8a99-4730e955600b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:59.414050 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa5f05-00d1-43f3-8a99-4730e955600b admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:59.414346 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa5f05-00d1-43f3-8a99-4730e955600b admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:59.414605 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa5f05-00d1-43f3-8a99-4730e955600b admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:59.415249 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fa5f05-00d1-43f3-8a99-4730e955600b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:59.417025 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fa5f05-00d1-43f3-8a99-4730e955600b admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:59.428473 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1281/2575] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:28:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:28:59.432784 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6be77d3-ff96-494f-936e-bd51f27fcc2b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:59.443114 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6be77d3-ff96-494f-936e-bd51f27fcc2b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:59.448058 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4dc589-44bf-47d1-b7a6-933541572ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:59.448797 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4dc589-44bf-47d1-b7a6-933541572ef1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:59.449088 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4dc589-44bf-47d1-b7a6-933541572ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:59.449360 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4dc589-44bf-47d1-b7a6-933541572ef1 admin admin] PATH_INFO: `/v3/projects/90a4080169cf46a699131eee62450822/users/6732bb239a134c6ba9908493609d4341/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:59.465605 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4dc589-44bf-47d1-b7a6-933541572ef1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90a4080169cf46a699131eee62450822, user_id=6732bb239a134c6ba9908493609d4341, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:59.469379 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4dc589-44bf-47d1-b7a6-933541572ef1 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:59.483959 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1295/2576] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:28:59 2022] PUT /identity/v3/projects/90a4080169cf46a699131eee62450822/users/6732bb239a134c6ba9908493609d4341/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:28:59.488167 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fa5f05-00d1-43f3-8a99-4730e955600b admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:59.498517 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fa5f05-00d1-43f3-8a99-4730e955600b admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:59.556615 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b72664-cb14-4f6a-81bb-16a842c3c70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:59.557257 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b72664-cb14-4f6a-81bb-16a842c3c70d admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:59.557537 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b72664-cb14-4f6a-81bb-16a842c3c70d admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:59.557807 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b72664-cb14-4f6a-81bb-16a842c3c70d admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:59.558516 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b72664-cb14-4f6a-81bb-16a842c3c70d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:59.560144 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b72664-cb14-4f6a-81bb-16a842c3c70d admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:59.570271 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1282/2577] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:28:59 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 05:28:59.574536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4dc589-44bf-47d1-b7a6-933541572ef1 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:59.585317 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4dc589-44bf-47d1-b7a6-933541572ef1 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:59.590152 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed4ab01-0d65-46e1-8bbb-e33b926cd655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:59.590803 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed4ab01-0d65-46e1-8bbb-e33b926cd655 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:59.591077 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed4ab01-0d65-46e1-8bbb-e33b926cd655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:59.591345 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed4ab01-0d65-46e1-8bbb-e33b926cd655 admin admin] PATH_INFO: `/v3/projects/90a4080169cf46a699131eee62450822/users/6732bb239a134c6ba9908493609d4341/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:59.593252 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed4ab01-0d65-46e1-8bbb-e33b926cd655 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90a4080169cf46a699131eee62450822, user_id=6732bb239a134c6ba9908493609d4341, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:59.596308 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed4ab01-0d65-46e1-8bbb-e33b926cd655 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:59.608931 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1296/2578] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:28:59 2022] PUT /identity/v3/projects/90a4080169cf46a699131eee62450822/users/6732bb239a134c6ba9908493609d4341/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:28:59.625413 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f78575-5f98-42d1-9f34-27a31c8625f6 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:59.625882 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f78575-5f98-42d1-9f34-27a31c8625f6 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:59.626282 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f78575-5f98-42d1-9f34-27a31c8625f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:59.656301 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-91f78575-5f98-42d1-9f34-27a31c8625f6 None None] MFA Rules not processed for user `6732bb239a134c6ba9908493609d4341`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:28:59.657672 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-91f78575-5f98-42d1-9f34-27a31c8625f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:59.736284 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1283/2579] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:28:59 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:28:59.749145 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed4ab01-0d65-46e1-8bbb-e33b926cd655 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:28:59.751127 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3ed4ab01-0d65-46e1-8bbb-e33b926cd655 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:28:59.812260 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed4ab01-0d65-46e1-8bbb-e33b926cd655 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:28:59.815928 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b8d5a-bb0a-44a8-a443-674fabb84ded service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:28:59.816541 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b8d5a-bb0a-44a8-a443-674fabb84ded service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:28:59.816813 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b8d5a-bb0a-44a8-a443-674fabb84ded service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:28:59.817076 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b8d5a-bb0a-44a8-a443-674fabb84ded service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:28:59.873299 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4b8d5a-bb0a-44a8-a443-674fabb84ded service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:28:59.874781 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4b8d5a-bb0a-44a8-a443-674fabb84ded service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:28:59.879409 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:28:59.884377 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1297/2580] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:28:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:29:00.036543 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f78575-5f98-42d1-9f34-27a31c8625f6 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:00.038767 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-91f78575-5f98-42d1-9f34-27a31c8625f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:00.055079 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:00.107558 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f78575-5f98-42d1-9f34-27a31c8625f6 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:00.111279 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65e764-a84f-4f4f-b553-99ca6748e5c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:00.111894 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65e764-a84f-4f4f-b553-99ca6748e5c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:00.112167 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65e764-a84f-4f4f-b553-99ca6748e5c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:00.112432 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65e764-a84f-4f4f-b553-99ca6748e5c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:00.114955 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-9c65e764-a84f-4f4f-b553-99ca6748e5c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:00.216920 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c65e764-a84f-4f4f-b553-99ca6748e5c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:00.219367 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c65e764-a84f-4f4f-b553-99ca6748e5c7 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:00.232277 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1284/2581] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:29:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:29:01.468496 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b8d5a-bb0a-44a8-a443-674fabb84ded service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:01.470485 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7c4b8d5a-bb0a-44a8-a443-674fabb84ded service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:01.481846 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b8d5a-bb0a-44a8-a443-674fabb84ded service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:01.485785 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aadda0-dfcd-472c-b222-860a4e4b33a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:01.486539 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aadda0-dfcd-472c-b222-860a4e4b33a1 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:01.486850 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aadda0-dfcd-472c-b222-860a4e4b33a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:01.487121 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aadda0-dfcd-472c-b222-860a4e4b33a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:01.550560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aadda0-dfcd-472c-b222-860a4e4b33a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:01.552062 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aadda0-dfcd-472c-b222-860a4e4b33a1 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:01.561181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1298/2582] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:29:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:29:02.143476 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65e764-a84f-4f4f-b553-99ca6748e5c7 service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:02.153867 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65e764-a84f-4f4f-b553-99ca6748e5c7 service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:02.157609 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ee3f9-0fdf-4baa-b5b3-415bd63d5f8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:02.158328 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ee3f9-0fdf-4baa-b5b3-415bd63d5f8f service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:02.158608 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ee3f9-0fdf-4baa-b5b3-415bd63d5f8f service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:02.158915 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ee3f9-0fdf-4baa-b5b3-415bd63d5f8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:02.172709 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8ee3f9-0fdf-4baa-b5b3-415bd63d5f8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:02.174299 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8ee3f9-0fdf-4baa-b5b3-415bd63d5f8f service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:02.186798 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1285/2583] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:29:02 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:29:05.576830 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aadda0-dfcd-472c-b222-860a4e4b33a1 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:05.593968 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aadda0-dfcd-472c-b222-860a4e4b33a1 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:05.599329 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9999d7c-c324-441b-a70c-1d3f5dc131a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:05.599929 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9999d7c-c324-441b-a70c-1d3f5dc131a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:05.600206 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9999d7c-c324-441b-a70c-1d3f5dc131a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:05.600471 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9999d7c-c324-441b-a70c-1d3f5dc131a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:05.602599 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e9999d7c-c324-441b-a70c-1d3f5dc131a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:05.613285 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9999d7c-c324-441b-a70c-1d3f5dc131a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:05.614757 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9999d7c-c324-441b-a70c-1d3f5dc131a6 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:05.623339 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1299/2584] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:29:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:29:06.098629 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ee3f9-0fdf-4baa-b5b3-415bd63d5f8f service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:06.164674 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ee3f9-0fdf-4baa-b5b3-415bd63d5f8f service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:06.169331 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594e51b0-17f8-4109-8851-035d66f5c53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:06.169966 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e51b0-17f8-4109-8851-035d66f5c53d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:06.170275 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e51b0-17f8-4109-8851-035d66f5c53d admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:06.170542 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e51b0-17f8-4109-8851-035d66f5c53d admin admin] PATH_INFO: `/v3/users/fb6a643b04624511b4b7fa5d0cea152f` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:06.171649 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594e51b0-17f8-4109-8851-035d66f5c53d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb6a643b04624511b4b7fa5d0cea152f)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:06.173599 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594e51b0-17f8-4109-8851-035d66f5c53d admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:06.216280 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:06.223053 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-594e51b0-17f8-4109-8851-035d66f5c53d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb6a643b04624511b4b7fa5d0cea152f'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:06.230592 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-594e51b0-17f8-4109-8851-035d66f5c53d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb6a643b04624511b4b7fa5d0cea152f'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:06.231320 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-594e51b0-17f8-4109-8851-035d66f5c53d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb6a643b04624511b4b7fa5d0cea152f'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:06.240445 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-594e51b0-17f8-4109-8851-035d66f5c53d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb6a643b04624511b4b7fa5d0cea152f'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:06.241105 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-594e51b0-17f8-4109-8851-035d66f5c53d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb6a643b04624511b4b7fa5d0cea152f'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:06.258177 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1286/2585] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:29:06 2022] DELETE /identity/v3/users/fb6a643b04624511b4b7fa5d0cea152f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:06.262865 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c4049-78cd-4258-88e5-d75e4798bb0e None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:06.263146 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c4049-78cd-4258-88e5-d75e4798bb0e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:06.263421 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c4049-78cd-4258-88e5-d75e4798bb0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:06.284282 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-153c4049-78cd-4258-88e5-d75e4798bb0e None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:29:06.285766 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-153c4049-78cd-4258-88e5-d75e4798bb0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:06.345925 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1300/2586] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:29:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:29:06.355935 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594e51b0-17f8-4109-8851-035d66f5c53d admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:06.357337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-594e51b0-17f8-4109-8851-035d66f5c53d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:06.416414 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594e51b0-17f8-4109-8851-035d66f5c53d admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:06.420557 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500301c2-cde1-477b-bf73-6d28d905baba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:06.421192 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500301c2-cde1-477b-bf73-6d28d905baba service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:06.421466 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500301c2-cde1-477b-bf73-6d28d905baba service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:06.421731 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500301c2-cde1-477b-bf73-6d28d905baba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:06.431103 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500301c2-cde1-477b-bf73-6d28d905baba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:06.432574 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500301c2-cde1-477b-bf73-6d28d905baba service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:06.442202 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1287/2587] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:29:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:29:06.712930 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c4049-78cd-4258-88e5-d75e4798bb0e None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:06.714493 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-153c4049-78cd-4258-88e5-d75e4798bb0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:06.727648 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c4049-78cd-4258-88e5-d75e4798bb0e None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:06.732778 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdd518c-2ba2-4a09-ab30-7662652302c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:06.733418 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd518c-2ba2-4a09-ab30-7662652302c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:06.733742 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd518c-2ba2-4a09-ab30-7662652302c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:06.734049 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd518c-2ba2-4a09-ab30-7662652302c7 admin admin] PATH_INFO: `/v3/projects/4a8f7f9c72f54b9ebd2f93bde3d6d8dc` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:06.735300 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdd518c-2ba2-4a09-ab30-7662652302c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a8f7f9c72f54b9ebd2f93bde3d6d8dc)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:06.737378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdd518c-2ba2-4a09-ab30-7662652302c7 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:06.787372 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6bdd518c-2ba2-4a09-ab30-7662652302c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a8f7f9c72f54b9ebd2f93bde3d6d8dc', 'request_id': 'req-6bdd518c-2ba2-4a09-ab30-7662652302c7'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:06.793854 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6bdd518c-2ba2-4a09-ab30-7662652302c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a8f7f9c72f54b9ebd2f93bde3d6d8dc', 'request_id': 'req-6bdd518c-2ba2-4a09-ab30-7662652302c7'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:06.803818 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6bdd518c-2ba2-4a09-ab30-7662652302c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a8f7f9c72f54b9ebd2f93bde3d6d8dc', 'request_id': 'req-6bdd518c-2ba2-4a09-ab30-7662652302c7'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:06.815054 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6bdd518c-2ba2-4a09-ab30-7662652302c7 admin admin] The token cache is being invalidate because project 4a8f7f9c72f54b9ebd2f93bde3d6d8dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:29:06.815335 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6bdd518c-2ba2-4a09-ab30-7662652302c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:06.816092 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6bdd518c-2ba2-4a09-ab30-7662652302c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:06.817670 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1301/2588] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:29:06 2022] DELETE /identity/v3/projects/4a8f7f9c72f54b9ebd2f93bde3d6d8dc => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:06.822157 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500301c2-cde1-477b-bf73-6d28d905baba service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:06.832219 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500301c2-cde1-477b-bf73-6d28d905baba service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:06.893261 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef15dc7f-50a2-4975-b18c-03ab254cbf90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:06.893805 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15dc7f-50a2-4975-b18c-03ab254cbf90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:06.894121 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15dc7f-50a2-4975-b18c-03ab254cbf90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:06.894376 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15dc7f-50a2-4975-b18c-03ab254cbf90 admin admin] PATH_INFO: `/v3/users/37f1f5efa55c4d488a16b9832cf2e0ef` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:06.895587 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef15dc7f-50a2-4975-b18c-03ab254cbf90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37f1f5efa55c4d488a16b9832cf2e0ef)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:06.897614 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef15dc7f-50a2-4975-b18c-03ab254cbf90 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:06.947233 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ef15dc7f-50a2-4975-b18c-03ab254cbf90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37f1f5efa55c4d488a16b9832cf2e0ef'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:06.954404 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ef15dc7f-50a2-4975-b18c-03ab254cbf90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37f1f5efa55c4d488a16b9832cf2e0ef'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:06.955146 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ef15dc7f-50a2-4975-b18c-03ab254cbf90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37f1f5efa55c4d488a16b9832cf2e0ef'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:06.964342 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ef15dc7f-50a2-4975-b18c-03ab254cbf90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37f1f5efa55c4d488a16b9832cf2e0ef'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:06.965083 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ef15dc7f-50a2-4975-b18c-03ab254cbf90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37f1f5efa55c4d488a16b9832cf2e0ef'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:06.966387 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:06.985696 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1288/2589] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:29:06 2022] DELETE /identity/v3/users/37f1f5efa55c4d488a16b9832cf2e0ef => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:07.266478 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdd518c-2ba2-4a09-ab30-7662652302c7 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:07.268168 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6bdd518c-2ba2-4a09-ab30-7662652302c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:07.283733 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdd518c-2ba2-4a09-ab30-7662652302c7 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:07.290364 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:07.351663 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc25bac-15ce-43fa-a258-55f2422fb453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:07.352454 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc25bac-15ce-43fa-a258-55f2422fb453 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:07.352739 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc25bac-15ce-43fa-a258-55f2422fb453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:07.353007 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc25bac-15ce-43fa-a258-55f2422fb453 admin admin] PATH_INFO: `/v3/projects/5559d3d0435d42339fb208420aba40a0` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:07.354192 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc25bac-15ce-43fa-a258-55f2422fb453 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5559d3d0435d42339fb208420aba40a0)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:07.356110 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc25bac-15ce-43fa-a258-55f2422fb453 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:07.399713 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef15dc7f-50a2-4975-b18c-03ab254cbf90 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:07.401373 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-ef15dc7f-50a2-4975-b18c-03ab254cbf90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:07.411193 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0cc25bac-15ce-43fa-a258-55f2422fb453 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5559d3d0435d42339fb208420aba40a0', 'request_id': 'req-0cc25bac-15ce-43fa-a258-55f2422fb453'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:07.420376 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0cc25bac-15ce-43fa-a258-55f2422fb453 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5559d3d0435d42339fb208420aba40a0', 'request_id': 'req-0cc25bac-15ce-43fa-a258-55f2422fb453'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:07.429048 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0cc25bac-15ce-43fa-a258-55f2422fb453 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5559d3d0435d42339fb208420aba40a0', 'request_id': 'req-0cc25bac-15ce-43fa-a258-55f2422fb453'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:07.439640 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0cc25bac-15ce-43fa-a258-55f2422fb453 admin admin] The token cache is being invalidate because project 5559d3d0435d42339fb208420aba40a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:29:07.440031 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0cc25bac-15ce-43fa-a258-55f2422fb453 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:07.440644 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0cc25bac-15ce-43fa-a258-55f2422fb453 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:07.442398 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1302/2590] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:29:07 2022] DELETE /identity/v3/projects/5559d3d0435d42339fb208420aba40a0 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:07.446172 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc25bac-15ce-43fa-a258-55f2422fb453 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:07.447482 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0cc25bac-15ce-43fa-a258-55f2422fb453 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:07.466880 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef15dc7f-50a2-4975-b18c-03ab254cbf90 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:07.509116 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc25bac-15ce-43fa-a258-55f2422fb453 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:07.514090 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8e03c-1f90-49d4-b84d-4cfff76c2b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:07.514720 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8e03c-1f90-49d4-b84d-4cfff76c2b3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:07.514966 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae3060f-7290-4758-9928-d2a4353f8596 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:07.515582 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3060f-7290-4758-9928-d2a4353f8596 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:07.515775 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8e03c-1f90-49d4-b84d-4cfff76c2b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:07.515883 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3060f-7290-4758-9928-d2a4353f8596 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:07.516050 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8e03c-1f90-49d4-b84d-4cfff76c2b3f admin admin] PATH_INFO: `/v3/users/6732bb239a134c6ba9908493609d4341` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:07.516157 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3060f-7290-4758-9928-d2a4353f8596 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:07.517178 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8e03c-1f90-49d4-b84d-4cfff76c2b3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6732bb239a134c6ba9908493609d4341)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:07.517830 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-eae3060f-7290-4758-9928-d2a4353f8596 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:07.519125 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8e03c-1f90-49d4-b84d-4cfff76c2b3f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:07.560144 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:07.566928 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-75c8e03c-1f90-49d4-b84d-4cfff76c2b3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6732bb239a134c6ba9908493609d4341'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:07.575074 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-75c8e03c-1f90-49d4-b84d-4cfff76c2b3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6732bb239a134c6ba9908493609d4341'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:07.583389 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-75c8e03c-1f90-49d4-b84d-4cfff76c2b3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6732bb239a134c6ba9908493609d4341'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:07.584076 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-75c8e03c-1f90-49d4-b84d-4cfff76c2b3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6732bb239a134c6ba9908493609d4341'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:07.596916 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-75c8e03c-1f90-49d4-b84d-4cfff76c2b3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6732bb239a134c6ba9908493609d4341'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:07.601223 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae3060f-7290-4758-9928-d2a4353f8596 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:07.602758 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae3060f-7290-4758-9928-d2a4353f8596 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:07.602978 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1303/2591] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:29:07 2022] DELETE /identity/v3/users/6732bb239a134c6ba9908493609d4341 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:29:07.622800 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1289/2592] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:29:07 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 05:29:07.887647 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8e03c-1f90-49d4-b84d-4cfff76c2b3f admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:07.890082 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-75c8e03c-1f90-49d4-b84d-4cfff76c2b3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:07.973801 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8e03c-1f90-49d4-b84d-4cfff76c2b3f admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:07.978891 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e2c3d4-c3cb-4571-8baa-a5c1134d6add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:07.979537 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2c3d4-c3cb-4571-8baa-a5c1134d6add admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:07.979809 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2c3d4-c3cb-4571-8baa-a5c1134d6add admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:07.980071 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2c3d4-c3cb-4571-8baa-a5c1134d6add admin admin] PATH_INFO: `/v3/projects/90a4080169cf46a699131eee62450822` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:07.981461 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e2c3d4-c3cb-4571-8baa-a5c1134d6add admin admin] RBAC: Authorizing `identity:delete_project(project_id=90a4080169cf46a699131eee62450822)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:07.983688 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e2c3d4-c3cb-4571-8baa-a5c1134d6add admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:08.032688 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d2e2c3d4-c3cb-4571-8baa-a5c1134d6add admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90a4080169cf46a699131eee62450822', 'request_id': 'req-d2e2c3d4-c3cb-4571-8baa-a5c1134d6add'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:08.039052 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d2e2c3d4-c3cb-4571-8baa-a5c1134d6add admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90a4080169cf46a699131eee62450822', 'request_id': 'req-d2e2c3d4-c3cb-4571-8baa-a5c1134d6add'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:08.050802 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d2e2c3d4-c3cb-4571-8baa-a5c1134d6add admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90a4080169cf46a699131eee62450822', 'request_id': 'req-d2e2c3d4-c3cb-4571-8baa-a5c1134d6add'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:08.062422 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d2e2c3d4-c3cb-4571-8baa-a5c1134d6add admin admin] The token cache is being invalidate because project 90a4080169cf46a699131eee62450822 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:29:08.062808 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d2e2c3d4-c3cb-4571-8baa-a5c1134d6add admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:08.063511 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d2e2c3d4-c3cb-4571-8baa-a5c1134d6add admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:08.065092 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1304/2593] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:29:07 2022] DELETE /identity/v3/projects/90a4080169cf46a699131eee62450822 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:08.070291 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be476f-6404-4e23-92d8-90f313d8265e None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:08.070573 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be476f-6404-4e23-92d8-90f313d8265e None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:08.070841 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be476f-6404-4e23-92d8-90f313d8265e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:08.091206 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-06be476f-6404-4e23-92d8-90f313d8265e None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:29:08.092617 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-06be476f-6404-4e23-92d8-90f313d8265e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:08.117569 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:08.162874 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1290/2594] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:29:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:29:08.232164 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d44bf-f299-4927-87fc-254466624410 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:08.232464 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d44bf-f299-4927-87fc-254466624410 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:08.232734 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d44bf-f299-4927-87fc-254466624410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:08.254434 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-6f3d44bf-f299-4927-87fc-254466624410 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:29:08.255720 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6f3d44bf-f299-4927-87fc-254466624410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:08.264011 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1305/2595] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:29:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:29:08.268433 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be476f-6404-4e23-92d8-90f313d8265e None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:08.278519 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be476f-6404-4e23-92d8-90f313d8265e None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:08.283222 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6fa9a9-7618-4c76-8118-0741257a4cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:08.283845 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6fa9a9-7618-4c76-8118-0741257a4cda admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:08.284108 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6fa9a9-7618-4c76-8118-0741257a4cda admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:08.284374 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6fa9a9-7618-4c76-8118-0741257a4cda admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:08.285170 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6fa9a9-7618-4c76-8118-0741257a4cda admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:29:08.285470 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6fa9a9-7618-4c76-8118-0741257a4cda admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:08.287133 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6fa9a9-7618-4c76-8118-0741257a4cda admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:08.302268 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1291/2596] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:29:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:29:08.307319 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d44bf-f299-4927-87fc-254466624410 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:08.319826 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d44bf-f299-4927-87fc-254466624410 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:08.325123 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c59eae2-483b-4068-b67e-4bb9ed8b8d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:08.325735 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59eae2-483b-4068-b67e-4bb9ed8b8d36 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:08.326075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59eae2-483b-4068-b67e-4bb9ed8b8d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:08.326348 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59eae2-483b-4068-b67e-4bb9ed8b8d36 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:08.326983 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c59eae2-483b-4068-b67e-4bb9ed8b8d36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:08.328917 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c59eae2-483b-4068-b67e-4bb9ed8b8d36 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:08.360945 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1306/2597] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:29:08 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:29:08.365499 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6fa9a9-7618-4c76-8118-0741257a4cda admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:08.376104 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6fa9a9-7618-4c76-8118-0741257a4cda admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:08.434996 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511feef0-ba1e-4f1e-badc-75baf4cb0365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:08.435648 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511feef0-ba1e-4f1e-badc-75baf4cb0365 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:08.435932 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511feef0-ba1e-4f1e-badc-75baf4cb0365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:08.436205 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511feef0-ba1e-4f1e-badc-75baf4cb0365 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:08.436880 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511feef0-ba1e-4f1e-badc-75baf4cb0365 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:08.438794 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511feef0-ba1e-4f1e-badc-75baf4cb0365 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:08.461680 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1292/2598] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:29:08 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:29:08.466062 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c59eae2-483b-4068-b67e-4bb9ed8b8d36 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:08.477920 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c59eae2-483b-4068-b67e-4bb9ed8b8d36 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:08.483880 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1404e0-0f09-42eb-b555-852f8c7f5f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:08.484595 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1404e0-0f09-42eb-b555-852f8c7f5f89 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:08.484956 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1404e0-0f09-42eb-b555-852f8c7f5f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:08.485288 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1404e0-0f09-42eb-b555-852f8c7f5f89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:08.486027 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1404e0-0f09-42eb-b555-852f8c7f5f89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:08.487735 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1404e0-0f09-42eb-b555-852f8c7f5f89 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:08.499887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1307/2599] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:29:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:29:08.504291 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511feef0-ba1e-4f1e-badc-75baf4cb0365 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:08.515367 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511feef0-ba1e-4f1e-badc-75baf4cb0365 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:08.520342 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2b1313-b747-410f-8432-810f5be4bacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:08.520939 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b1313-b747-410f-8432-810f5be4bacc admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:08.521216 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b1313-b747-410f-8432-810f5be4bacc admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:08.521485 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b1313-b747-410f-8432-810f5be4bacc admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:08.522174 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2b1313-b747-410f-8432-810f5be4bacc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:08.523785 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2b1313-b747-410f-8432-810f5be4bacc admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:08.534952 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1293/2600] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:29:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:29:08.539306 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1404e0-0f09-42eb-b555-852f8c7f5f89 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:08.549750 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1404e0-0f09-42eb-b555-852f8c7f5f89 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:08.555368 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe04202-1902-4260-a8ee-4816a5a7a3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:08.556103 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe04202-1902-4260-a8ee-4816a5a7a3ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:08.556444 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe04202-1902-4260-a8ee-4816a5a7a3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:08.556775 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe04202-1902-4260-a8ee-4816a5a7a3ba admin admin] PATH_INFO: `/v3/projects/25348a2fd6ba43bbb3cf5e8126cd8a67/users/695bbf71deee465985055e79dc8e105e/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:08.560581 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:08.572534 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe04202-1902-4260-a8ee-4816a5a7a3ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=25348a2fd6ba43bbb3cf5e8126cd8a67, user_id=695bbf71deee465985055e79dc8e105e, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:08.575802 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe04202-1902-4260-a8ee-4816a5a7a3ba admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:08.598290 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1308/2601] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:29:08 2022] PUT /identity/v3/projects/25348a2fd6ba43bbb3cf5e8126cd8a67/users/695bbf71deee465985055e79dc8e105e/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:08.603559 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0b5d4-51f2-4548-bf09-d0d4aa5a83a9 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:08.603850 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0b5d4-51f2-4548-bf09-d0d4aa5a83a9 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:08.604125 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0b5d4-51f2-4548-bf09-d0d4aa5a83a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:08.623345 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-90a0b5d4-51f2-4548-bf09-d0d4aa5a83a9 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:29:08.624545 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-90a0b5d4-51f2-4548-bf09-d0d4aa5a83a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:08.693410 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1294/2602] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:29:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:29:08.700692 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe04202-1902-4260-a8ee-4816a5a7a3ba admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:08.702561 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-abe04202-1902-4260-a8ee-4816a5a7a3ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:08.763135 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe04202-1902-4260-a8ee-4816a5a7a3ba admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:08.766980 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65157058-fd7d-4e1f-842b-f90678ee713b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:08.767581 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65157058-fd7d-4e1f-842b-f90678ee713b service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:08.767855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65157058-fd7d-4e1f-842b-f90678ee713b service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:08.768118 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65157058-fd7d-4e1f-842b-f90678ee713b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:08.777550 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65157058-fd7d-4e1f-842b-f90678ee713b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:08.779040 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65157058-fd7d-4e1f-842b-f90678ee713b service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:08.788851 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1309/2603] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:29:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:29:09.502229 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e51565-98a6-4210-9464-0341f476b57c None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:09.502770 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e51565-98a6-4210-9464-0341f476b57c None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:09.502770 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e51565-98a6-4210-9464-0341f476b57c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:09.533842 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-90e51565-98a6-4210-9464-0341f476b57c None None] MFA Rules not processed for user `695bbf71deee465985055e79dc8e105e`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:29:09.535149 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-90e51565-98a6-4210-9464-0341f476b57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:09.556270 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:09.612334 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1295/2604] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:29:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:29:09.625850 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65157058-fd7d-4e1f-842b-f90678ee713b service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:09.627486 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-65157058-fd7d-4e1f-842b-f90678ee713b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:09.698506 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65157058-fd7d-4e1f-842b-f90678ee713b service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:09.702390 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb925b-a057-4a93-91c7-5af0062a20fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:09.703077 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb925b-a057-4a93-91c7-5af0062a20fb service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:09.703357 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb925b-a057-4a93-91c7-5af0062a20fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:09.703705 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb925b-a057-4a93-91c7-5af0062a20fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:09.713622 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbb925b-a057-4a93-91c7-5af0062a20fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:09.715168 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbb925b-a057-4a93-91c7-5af0062a20fb service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:09.725366 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1310/2605] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:29:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 05:29:11.101464 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649cc4fb-64d7-46d5-8d93-92b76b330e2f None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:11.102127 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649cc4fb-64d7-46d5-8d93-92b76b330e2f None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:11.102235 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649cc4fb-64d7-46d5-8d93-92b76b330e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:11.121672 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-649cc4fb-64d7-46d5-8d93-92b76b330e2f None None] MFA Rules not processed for user `695bbf71deee465985055e79dc8e105e`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:29:11.123043 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-649cc4fb-64d7-46d5-8d93-92b76b330e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:11.130449 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1296/2606] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:29:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:29:16.302199 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb925b-a057-4a93-91c7-5af0062a20fb service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:16.304184 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-dcbb925b-a057-4a93-91c7-5af0062a20fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:16.315966 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb925b-a057-4a93-91c7-5af0062a20fb service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:16.319606 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c8670e-4777-476a-95e7-5c24f0f1f1b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:16.320203 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8670e-4777-476a-95e7-5c24f0f1f1b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:16.320743 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8670e-4777-476a-95e7-5c24f0f1f1b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:16.320743 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8670e-4777-476a-95e7-5c24f0f1f1b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:16.322477 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-50c8670e-4777-476a-95e7-5c24f0f1f1b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:16.334956 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c8670e-4777-476a-95e7-5c24f0f1f1b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:16.336357 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c8670e-4777-476a-95e7-5c24f0f1f1b6 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:16.347375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1311/2607] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:29:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 05:29:20.057417 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649cc4fb-64d7-46d5-8d93-92b76b330e2f None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:20.058999 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-649cc4fb-64d7-46d5-8d93-92b76b330e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:20.075318 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649cc4fb-64d7-46d5-8d93-92b76b330e2f None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:20.081404 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23cf56b-414d-4023-bebc-6835cb0ecf0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:20.082082 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cf56b-414d-4023-bebc-6835cb0ecf0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:20.082356 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cf56b-414d-4023-bebc-6835cb0ecf0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:20.082619 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cf56b-414d-4023-bebc-6835cb0ecf0c admin admin] PATH_INFO: `/v3/users/af50ca6347404cdf8eb213d27053af16` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:20.083829 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23cf56b-414d-4023-bebc-6835cb0ecf0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=af50ca6347404cdf8eb213d27053af16)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:20.086199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23cf56b-414d-4023-bebc-6835cb0ecf0c admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:20.135448 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c23cf56b-414d-4023-bebc-6835cb0ecf0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af50ca6347404cdf8eb213d27053af16'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:20.142014 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c23cf56b-414d-4023-bebc-6835cb0ecf0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af50ca6347404cdf8eb213d27053af16'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:20.142697 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c23cf56b-414d-4023-bebc-6835cb0ecf0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af50ca6347404cdf8eb213d27053af16'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:20.151960 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c23cf56b-414d-4023-bebc-6835cb0ecf0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af50ca6347404cdf8eb213d27053af16'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:20.153043 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c23cf56b-414d-4023-bebc-6835cb0ecf0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af50ca6347404cdf8eb213d27053af16'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:20.170115 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1297/2608] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:29:20 2022] DELETE /identity/v3/users/af50ca6347404cdf8eb213d27053af16 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:20.350139 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c8670e-4777-476a-95e7-5c24f0f1f1b6 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:20.360982 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c8670e-4777-476a-95e7-5c24f0f1f1b6 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:20.419339 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737f1752-22c6-47c1-a7c7-48df507a4f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:20.419893 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f1752-22c6-47c1-a7c7-48df507a4f82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:20.420180 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f1752-22c6-47c1-a7c7-48df507a4f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:20.420445 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f1752-22c6-47c1-a7c7-48df507a4f82 admin admin] PATH_INFO: `/v3/projects/164c57b95c274f89bd391cf111efa374` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:20.421579 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737f1752-22c6-47c1-a7c7-48df507a4f82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=164c57b95c274f89bd391cf111efa374)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:20.423551 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737f1752-22c6-47c1-a7c7-48df507a4f82 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:20.444058 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:20.473448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-737f1752-22c6-47c1-a7c7-48df507a4f82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '164c57b95c274f89bd391cf111efa374', 'request_id': 'req-737f1752-22c6-47c1-a7c7-48df507a4f82'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:20.479602 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-737f1752-22c6-47c1-a7c7-48df507a4f82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '164c57b95c274f89bd391cf111efa374', 'request_id': 'req-737f1752-22c6-47c1-a7c7-48df507a4f82'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:20.488232 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-737f1752-22c6-47c1-a7c7-48df507a4f82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '164c57b95c274f89bd391cf111efa374', 'request_id': 'req-737f1752-22c6-47c1-a7c7-48df507a4f82'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:20.499093 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-737f1752-22c6-47c1-a7c7-48df507a4f82 admin admin] The token cache is being invalidate because project 164c57b95c274f89bd391cf111efa374 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:29:20.499462 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-737f1752-22c6-47c1-a7c7-48df507a4f82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:20.500126 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-737f1752-22c6-47c1-a7c7-48df507a4f82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:20.501722 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1312/2609] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:29:20 2022] DELETE /identity/v3/projects/164c57b95c274f89bd391cf111efa374 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:20.507695 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f3de4-197d-42f1-8b9f-1a1ff57c3300 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:20.507981 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f3de4-197d-42f1-8b9f-1a1ff57c3300 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:20.508256 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f3de4-197d-42f1-8b9f-1a1ff57c3300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:20.530963 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-739f3de4-197d-42f1-8b9f-1a1ff57c3300 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:29:20.532191 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-739f3de4-197d-42f1-8b9f-1a1ff57c3300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:20.599280 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1298/2610] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:29:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:29:20.607885 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff988631-6fbf-44f6-8ab0-899084d2c575 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:20.608173 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff988631-6fbf-44f6-8ab0-899084d2c575 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:20.608467 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff988631-6fbf-44f6-8ab0-899084d2c575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:20.630870 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ff988631-6fbf-44f6-8ab0-899084d2c575 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:29:20.632073 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ff988631-6fbf-44f6-8ab0-899084d2c575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:20.642998 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1313/2611] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:29:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:29:20.648064 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f3de4-197d-42f1-8b9f-1a1ff57c3300 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:20.660778 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f3de4-197d-42f1-8b9f-1a1ff57c3300 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:20.665590 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c44d357-3834-447a-bad1-5aa3c96db58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:20.666235 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44d357-3834-447a-bad1-5aa3c96db58f admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:20.666510 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44d357-3834-447a-bad1-5aa3c96db58f admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:20.666774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44d357-3834-447a-bad1-5aa3c96db58f admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:20.667419 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c44d357-3834-447a-bad1-5aa3c96db58f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:29:20.667714 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c44d357-3834-447a-bad1-5aa3c96db58f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:20.669396 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c44d357-3834-447a-bad1-5aa3c96db58f admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:20.682425 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1299/2612] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:29:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:29:20.686829 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff988631-6fbf-44f6-8ab0-899084d2c575 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:20.696299 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff988631-6fbf-44f6-8ab0-899084d2c575 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:20.701711 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deb0d56-4e64-46ab-9dfc-cdd39a0f3be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:20.702505 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb0d56-4e64-46ab-9dfc-cdd39a0f3be4 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:20.702760 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb0d56-4e64-46ab-9dfc-cdd39a0f3be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:20.703014 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb0d56-4e64-46ab-9dfc-cdd39a0f3be4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:20.703710 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deb0d56-4e64-46ab-9dfc-cdd39a0f3be4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:20.705620 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deb0d56-4e64-46ab-9dfc-cdd39a0f3be4 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:20.736153 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1314/2613] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:29:20 2022] POST /identity/v3/projects => generated 363 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:29:20.740465 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c44d357-3834-447a-bad1-5aa3c96db58f admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:20.760451 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c44d357-3834-447a-bad1-5aa3c96db58f admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:20.770780 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:20.822037 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0866992-c495-4c1b-9d2b-b21ca97d6dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:20.822691 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0866992-c495-4c1b-9d2b-b21ca97d6dcb admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:20.822969 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0866992-c495-4c1b-9d2b-b21ca97d6dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:20.823236 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0866992-c495-4c1b-9d2b-b21ca97d6dcb admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:20.823869 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0866992-c495-4c1b-9d2b-b21ca97d6dcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:20.825841 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0866992-c495-4c1b-9d2b-b21ca97d6dcb admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:20.853210 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1300/2614] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:29:20 2022] POST /identity/v3/users => generated 282 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:29:20.858170 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deb0d56-4e64-46ab-9dfc-cdd39a0f3be4 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:20.870286 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deb0d56-4e64-46ab-9dfc-cdd39a0f3be4 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:20.875478 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6825cc80-40f6-4996-91fc-6a7cb0298d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:20.876070 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825cc80-40f6-4996-91fc-6a7cb0298d73 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:20.876347 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825cc80-40f6-4996-91fc-6a7cb0298d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:20.876630 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825cc80-40f6-4996-91fc-6a7cb0298d73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:20.877269 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6825cc80-40f6-4996-91fc-6a7cb0298d73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:20.878874 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6825cc80-40f6-4996-91fc-6a7cb0298d73 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:20.889060 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1315/2615] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:29:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:29:20.892890 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0866992-c495-4c1b-9d2b-b21ca97d6dcb admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:20.903172 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0866992-c495-4c1b-9d2b-b21ca97d6dcb admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:20.908774 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac632cc4-2df6-4de9-b591-971347de114e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:20.909368 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac632cc4-2df6-4de9-b591-971347de114e admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:20.909641 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac632cc4-2df6-4de9-b591-971347de114e admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:20.909906 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac632cc4-2df6-4de9-b591-971347de114e admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:20.910579 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac632cc4-2df6-4de9-b591-971347de114e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:20.912193 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac632cc4-2df6-4de9-b591-971347de114e admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:20.923230 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1301/2616] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:29:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:29:20.927536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6825cc80-40f6-4996-91fc-6a7cb0298d73 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:20.937580 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6825cc80-40f6-4996-91fc-6a7cb0298d73 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:20.942427 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce366ce5-fc85-469e-a2c4-fc469050f734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:20.943061 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce366ce5-fc85-469e-a2c4-fc469050f734 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:20.943333 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce366ce5-fc85-469e-a2c4-fc469050f734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:20.943581 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce366ce5-fc85-469e-a2c4-fc469050f734 admin admin] PATH_INFO: `/v3/projects/f16217f3b43846beb1731727299ac1ba/users/2dc8700d91db44bda3daa4ad7ca3e3b3/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:20.958801 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce366ce5-fc85-469e-a2c4-fc469050f734 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f16217f3b43846beb1731727299ac1ba, user_id=2dc8700d91db44bda3daa4ad7ca3e3b3, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:20.961510 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce366ce5-fc85-469e-a2c4-fc469050f734 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:20.978090 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1316/2617] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:29:20 2022] PUT /identity/v3/projects/f16217f3b43846beb1731727299ac1ba/users/2dc8700d91db44bda3daa4ad7ca3e3b3/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:20.988745 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5a1c3-ba7d-41c1-ba28-079a8908f8f1 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:20.989028 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5a1c3-ba7d-41c1-ba28-079a8908f8f1 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:20.989285 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5a1c3-ba7d-41c1-ba28-079a8908f8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:21.018221 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-35b5a1c3-ba7d-41c1-ba28-079a8908f8f1 None None] MFA Rules not processed for user `2dc8700d91db44bda3daa4ad7ca3e3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:29:21.019457 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-35b5a1c3-ba7d-41c1-ba28-079a8908f8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:21.084602 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1302/2618] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:29:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:29:21.089433 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce366ce5-fc85-469e-a2c4-fc469050f734 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:21.158031 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce366ce5-fc85-469e-a2c4-fc469050f734 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:21.164026 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af9e201-f317-4db5-912a-57b5b1dcd331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:21.164642 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af9e201-f317-4db5-912a-57b5b1dcd331 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:21.164913 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af9e201-f317-4db5-912a-57b5b1dcd331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:21.165173 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af9e201-f317-4db5-912a-57b5b1dcd331 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:21.165831 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af9e201-f317-4db5-912a-57b5b1dcd331 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:21.167768 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af9e201-f317-4db5-912a-57b5b1dcd331 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:21.197607 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1317/2619] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:29:21 2022] POST /identity/v3/projects => generated 365 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:29:21.201884 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b5a1c3-ba7d-41c1-ba28-079a8908f8f1 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:21.213081 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b5a1c3-ba7d-41c1-ba28-079a8908f8f1 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:21.277386 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2931bf8d-b878-4ad7-a263-87479a7c30ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:21.277987 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931bf8d-b878-4ad7-a263-87479a7c30ab admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:21.278301 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931bf8d-b878-4ad7-a263-87479a7c30ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:21.278561 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931bf8d-b878-4ad7-a263-87479a7c30ab admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:21.279188 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2931bf8d-b878-4ad7-a263-87479a7c30ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:21.281107 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2931bf8d-b878-4ad7-a263-87479a7c30ab admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:21.304346 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1303/2620] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:29:21 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:29:21.308475 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af9e201-f317-4db5-912a-57b5b1dcd331 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:21.318882 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af9e201-f317-4db5-912a-57b5b1dcd331 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:21.323433 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8486d289-d0a6-42d4-99d5-8b34c76444e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:21.324025 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486d289-d0a6-42d4-99d5-8b34c76444e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:21.324303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486d289-d0a6-42d4-99d5-8b34c76444e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:21.324570 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486d289-d0a6-42d4-99d5-8b34c76444e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:21.325209 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8486d289-d0a6-42d4-99d5-8b34c76444e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:21.326827 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8486d289-d0a6-42d4-99d5-8b34c76444e7 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:21.337490 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1318/2621] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:29:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:29:21.341547 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2931bf8d-b878-4ad7-a263-87479a7c30ab admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:21.352133 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2931bf8d-b878-4ad7-a263-87479a7c30ab admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:21.357203 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4f8d05-0e24-4e89-a018-391d94699949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:21.357848 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4f8d05-0e24-4e89-a018-391d94699949 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:21.358186 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4f8d05-0e24-4e89-a018-391d94699949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:21.358438 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4f8d05-0e24-4e89-a018-391d94699949 admin admin] PATH_INFO: `/v3/projects/32a9912afcdf4f6187037e171bac4d98/users/fa6ef8cd734845b6b306aca26c4c19eb/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:21.361538 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:21.375304 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4f8d05-0e24-4e89-a018-391d94699949 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32a9912afcdf4f6187037e171bac4d98, user_id=fa6ef8cd734845b6b306aca26c4c19eb, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:21.378061 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4f8d05-0e24-4e89-a018-391d94699949 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:21.392724 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1304/2622] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:29:21 2022] PUT /identity/v3/projects/32a9912afcdf4f6187037e171bac4d98/users/fa6ef8cd734845b6b306aca26c4c19eb/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:21.396737 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8486d289-d0a6-42d4-99d5-8b34c76444e7 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:21.407166 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8486d289-d0a6-42d4-99d5-8b34c76444e7 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:21.419816 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:21.466538 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69484024-bdd1-4411-a3c8-403f1928fc59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:21.467064 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69484024-bdd1-4411-a3c8-403f1928fc59 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:21.467340 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69484024-bdd1-4411-a3c8-403f1928fc59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:21.467608 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69484024-bdd1-4411-a3c8-403f1928fc59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:21.468558 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69484024-bdd1-4411-a3c8-403f1928fc59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:21.470182 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69484024-bdd1-4411-a3c8-403f1928fc59 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:21.480385 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1319/2623] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:29:21 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:29:21.484854 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4f8d05-0e24-4e89-a018-391d94699949 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:21.498654 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4f8d05-0e24-4e89-a018-391d94699949 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:21.504421 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8308de78-7615-4823-9747-10562c34fd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:21.505057 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8308de78-7615-4823-9747-10562c34fd36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:21.505346 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8308de78-7615-4823-9747-10562c34fd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:21.505611 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8308de78-7615-4823-9747-10562c34fd36 admin admin] PATH_INFO: `/v3/projects/32a9912afcdf4f6187037e171bac4d98/users/fa6ef8cd734845b6b306aca26c4c19eb/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:21.507770 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8308de78-7615-4823-9747-10562c34fd36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32a9912afcdf4f6187037e171bac4d98, user_id=fa6ef8cd734845b6b306aca26c4c19eb, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:21.510479 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8308de78-7615-4823-9747-10562c34fd36 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:21.520903 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1305/2624] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:29:21 2022] PUT /identity/v3/projects/32a9912afcdf4f6187037e171bac4d98/users/fa6ef8cd734845b6b306aca26c4c19eb/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:21.539262 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b96dc3-a7ca-4194-9fcf-5eee5d3fb4b5 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:21.539643 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b96dc3-a7ca-4194-9fcf-5eee5d3fb4b5 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:21.540034 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b96dc3-a7ca-4194-9fcf-5eee5d3fb4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:21.573820 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-03b96dc3-a7ca-4194-9fcf-5eee5d3fb4b5 None None] MFA Rules not processed for user `fa6ef8cd734845b6b306aca26c4c19eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:29:21.575106 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-03b96dc3-a7ca-4194-9fcf-5eee5d3fb4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:21.660048 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1320/2625] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:29:21 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:29:21.671876 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8308de78-7615-4823-9747-10562c34fd36 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:21.673683 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-8308de78-7615-4823-9747-10562c34fd36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:21.742548 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8308de78-7615-4823-9747-10562c34fd36 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:21.749501 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0008b912-e831-40b9-bd6b-3dd30f71ea41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:21.750466 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008b912-e831-40b9-bd6b-3dd30f71ea41 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:21.750847 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008b912-e831-40b9-bd6b-3dd30f71ea41 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:21.751161 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008b912-e831-40b9-bd6b-3dd30f71ea41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:21.762563 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0008b912-e831-40b9-bd6b-3dd30f71ea41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:21.764083 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0008b912-e831-40b9-bd6b-3dd30f71ea41 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:21.774052 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1306/2626] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:29:21 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:29:21.854729 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b96dc3-a7ca-4194-9fcf-5eee5d3fb4b5 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:21.866157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b96dc3-a7ca-4194-9fcf-5eee5d3fb4b5 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:21.870572 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ef3f7-c706-48cc-91a2-c90da7ec099c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:21.871178 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ef3f7-c706-48cc-91a2-c90da7ec099c service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:21.871447 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ef3f7-c706-48cc-91a2-c90da7ec099c service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:21.871715 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ef3f7-c706-48cc-91a2-c90da7ec099c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:21.933760 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450ef3f7-c706-48cc-91a2-c90da7ec099c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:21.935272 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450ef3f7-c706-48cc-91a2-c90da7ec099c service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:21.944601 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1321/2627] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:29:21 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:29:31.769661 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=37,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:31.940145 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=83 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:33.423690 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0008b912-e831-40b9-bd6b-3dd30f71ea41 service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:33.490163 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0008b912-e831-40b9-bd6b-3dd30f71ea41 service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:33.495804 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2e8b8-7f0b-431c-9cef-daf18600f9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:33.496532 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2e8b8-7f0b-431c-9cef-daf18600f9f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:33.496948 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2e8b8-7f0b-431c-9cef-daf18600f9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:33.497243 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2e8b8-7f0b-431c-9cef-daf18600f9f4 admin admin] PATH_INFO: `/v3/users/2dc8700d91db44bda3daa4ad7ca3e3b3` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:33.498699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c2e8b8-7f0b-431c-9cef-daf18600f9f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dc8700d91db44bda3daa4ad7ca3e3b3)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:33.500973 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c2e8b8-7f0b-431c-9cef-daf18600f9f4 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:33.551287 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-23c2e8b8-7f0b-431c-9cef-daf18600f9f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dc8700d91db44bda3daa4ad7ca3e3b3'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:33.557891 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-23c2e8b8-7f0b-431c-9cef-daf18600f9f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dc8700d91db44bda3daa4ad7ca3e3b3'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:33.558571 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-23c2e8b8-7f0b-431c-9cef-daf18600f9f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dc8700d91db44bda3daa4ad7ca3e3b3'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:33.571363 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-23c2e8b8-7f0b-431c-9cef-daf18600f9f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dc8700d91db44bda3daa4ad7ca3e3b3'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:33.571991 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-23c2e8b8-7f0b-431c-9cef-daf18600f9f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dc8700d91db44bda3daa4ad7ca3e3b3'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:33.591746 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1307/2628] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:29:33 2022] DELETE /identity/v3/users/2dc8700d91db44bda3daa4ad7ca3e3b3 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:33.597710 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45429500-6d56-4cf9-8c4f-1a4c4f5c6904 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:33.598439 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45429500-6d56-4cf9-8c4f-1a4c4f5c6904 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:33.598439 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45429500-6d56-4cf9-8c4f-1a4c4f5c6904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:33.618635 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-45429500-6d56-4cf9-8c4f-1a4c4f5c6904 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:29:33.619837 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-45429500-6d56-4cf9-8c4f-1a4c4f5c6904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:33.685142 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1322/2629] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:29:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:29:33.696084 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2e8b8-7f0b-431c-9cef-daf18600f9f4 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:33.697883 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-23c2e8b8-7f0b-431c-9cef-daf18600f9f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:33.759410 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2e8b8-7f0b-431c-9cef-daf18600f9f4 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:33.763583 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a5f6a-b4b5-418c-87df-35a63b7b06f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:33.764191 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a5f6a-b4b5-418c-87df-35a63b7b06f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:33.764471 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a5f6a-b4b5-418c-87df-35a63b7b06f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:33.764741 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a5f6a-b4b5-418c-87df-35a63b7b06f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:33.777325 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2a5f6a-b4b5-418c-87df-35a63b7b06f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:33.778818 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2a5f6a-b4b5-418c-87df-35a63b7b06f9 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:33.788647 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1308/2630] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:29:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:29:34.064974 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45429500-6d56-4cf9-8c4f-1a4c4f5c6904 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:34.066616 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-45429500-6d56-4cf9-8c4f-1a4c4f5c6904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:34.085173 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45429500-6d56-4cf9-8c4f-1a4c4f5c6904 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:34.094152 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dbf2d3-28f6-49b1-a455-5bcf17f95b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:34.094864 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dbf2d3-28f6-49b1-a455-5bcf17f95b27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:34.095161 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dbf2d3-28f6-49b1-a455-5bcf17f95b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:34.095449 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dbf2d3-28f6-49b1-a455-5bcf17f95b27 admin admin] PATH_INFO: `/v3/projects/f16217f3b43846beb1731727299ac1ba` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:34.096954 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dbf2d3-28f6-49b1-a455-5bcf17f95b27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f16217f3b43846beb1731727299ac1ba)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:34.099096 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dbf2d3-28f6-49b1-a455-5bcf17f95b27 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:34.150864 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e9dbf2d3-28f6-49b1-a455-5bcf17f95b27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f16217f3b43846beb1731727299ac1ba', 'request_id': 'req-e9dbf2d3-28f6-49b1-a455-5bcf17f95b27'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:34.157086 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e9dbf2d3-28f6-49b1-a455-5bcf17f95b27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f16217f3b43846beb1731727299ac1ba', 'request_id': 'req-e9dbf2d3-28f6-49b1-a455-5bcf17f95b27'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:34.167133 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e9dbf2d3-28f6-49b1-a455-5bcf17f95b27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f16217f3b43846beb1731727299ac1ba', 'request_id': 'req-e9dbf2d3-28f6-49b1-a455-5bcf17f95b27'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:34.178254 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e9dbf2d3-28f6-49b1-a455-5bcf17f95b27 admin admin] The token cache is being invalidate because project f16217f3b43846beb1731727299ac1ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:29:34.178541 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e9dbf2d3-28f6-49b1-a455-5bcf17f95b27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:34.179186 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e9dbf2d3-28f6-49b1-a455-5bcf17f95b27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:34.180712 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1323/2631] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:29:34 2022] DELETE /identity/v3/projects/f16217f3b43846beb1731727299ac1ba => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:34.184936 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a5f6a-b4b5-418c-87df-35a63b7b06f9 service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:34.195135 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a5f6a-b4b5-418c-87df-35a63b7b06f9 service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:34.214612 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:34.258278 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b395b2e-a0e8-4846-badd-40f7109b1e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:34.258940 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b395b2e-a0e8-4846-badd-40f7109b1e5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:34.259352 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b395b2e-a0e8-4846-badd-40f7109b1e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:34.259535 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b395b2e-a0e8-4846-badd-40f7109b1e5a admin admin] PATH_INFO: `/v3/users/fa6ef8cd734845b6b306aca26c4c19eb` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:34.260708 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b395b2e-a0e8-4846-badd-40f7109b1e5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa6ef8cd734845b6b306aca26c4c19eb)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:34.262796 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b395b2e-a0e8-4846-badd-40f7109b1e5a admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:34.313025 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0b395b2e-a0e8-4846-badd-40f7109b1e5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa6ef8cd734845b6b306aca26c4c19eb'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:34.321569 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0b395b2e-a0e8-4846-badd-40f7109b1e5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa6ef8cd734845b6b306aca26c4c19eb'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:34.322428 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0b395b2e-a0e8-4846-badd-40f7109b1e5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa6ef8cd734845b6b306aca26c4c19eb'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:34.333531 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0b395b2e-a0e8-4846-badd-40f7109b1e5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa6ef8cd734845b6b306aca26c4c19eb'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:34.334226 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0b395b2e-a0e8-4846-badd-40f7109b1e5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa6ef8cd734845b6b306aca26c4c19eb'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:34.352466 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1309/2632] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:29:34 2022] DELETE /identity/v3/users/fa6ef8cd734845b6b306aca26c4c19eb => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:34.636136 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dbf2d3-28f6-49b1-a455-5bcf17f95b27 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:34.637598 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e9dbf2d3-28f6-49b1-a455-5bcf17f95b27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:34.652499 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dbf2d3-28f6-49b1-a455-5bcf17f95b27 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:34.718229 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89f3fb-b224-43d2-a8c5-d2c75bb3d19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:34.719180 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89f3fb-b224-43d2-a8c5-d2c75bb3d19d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:34.719457 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89f3fb-b224-43d2-a8c5-d2c75bb3d19d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:34.719722 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89f3fb-b224-43d2-a8c5-d2c75bb3d19d admin admin] PATH_INFO: `/v3/projects/32a9912afcdf4f6187037e171bac4d98` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:34.720857 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89f3fb-b224-43d2-a8c5-d2c75bb3d19d admin admin] RBAC: Authorizing `identity:delete_project(project_id=32a9912afcdf4f6187037e171bac4d98)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:34.722814 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89f3fb-b224-43d2-a8c5-d2c75bb3d19d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:34.755301 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:34.773051 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0b89f3fb-b224-43d2-a8c5-d2c75bb3d19d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32a9912afcdf4f6187037e171bac4d98', 'request_id': 'req-0b89f3fb-b224-43d2-a8c5-d2c75bb3d19d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:34.780079 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0b89f3fb-b224-43d2-a8c5-d2c75bb3d19d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32a9912afcdf4f6187037e171bac4d98', 'request_id': 'req-0b89f3fb-b224-43d2-a8c5-d2c75bb3d19d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:34.789351 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0b89f3fb-b224-43d2-a8c5-d2c75bb3d19d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32a9912afcdf4f6187037e171bac4d98', 'request_id': 'req-0b89f3fb-b224-43d2-a8c5-d2c75bb3d19d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:34.799927 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0b89f3fb-b224-43d2-a8c5-d2c75bb3d19d admin admin] The token cache is being invalidate because project 32a9912afcdf4f6187037e171bac4d98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:29:34.800206 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0b89f3fb-b224-43d2-a8c5-d2c75bb3d19d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:34.800907 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0b89f3fb-b224-43d2-a8c5-d2c75bb3d19d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:34.805931 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1324/2633] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:29:34 2022] DELETE /identity/v3/projects/32a9912afcdf4f6187037e171bac4d98 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:34.811098 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd76a7b2-23fe-4806-8928-313f1ab73323 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:34.811603 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd76a7b2-23fe-4806-8928-313f1ab73323 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:34.811716 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd76a7b2-23fe-4806-8928-313f1ab73323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:34.833914 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-dd76a7b2-23fe-4806-8928-313f1ab73323 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:29:34.835086 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-dd76a7b2-23fe-4806-8928-313f1ab73323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:34.901594 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1310/2634] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:29:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:29:34.909800 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46eb5ca-8f01-4be2-bbff-3ddc6734b629 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:34.910121 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46eb5ca-8f01-4be2-bbff-3ddc6734b629 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:34.910431 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46eb5ca-8f01-4be2-bbff-3ddc6734b629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:34.932244 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e46eb5ca-8f01-4be2-bbff-3ddc6734b629 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:29:34.933587 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e46eb5ca-8f01-4be2-bbff-3ddc6734b629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:34.943131 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1325/2635] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:29:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:29:34.947521 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd76a7b2-23fe-4806-8928-313f1ab73323 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:34.957367 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd76a7b2-23fe-4806-8928-313f1ab73323 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:34.962813 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd2c246-3ba0-463b-ac38-68d081aa16ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:34.963420 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2c246-3ba0-463b-ac38-68d081aa16ad admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:34.963702 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2c246-3ba0-463b-ac38-68d081aa16ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:34.964011 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2c246-3ba0-463b-ac38-68d081aa16ad admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:34.964686 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd2c246-3ba0-463b-ac38-68d081aa16ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:29:34.965000 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd2c246-3ba0-463b-ac38-68d081aa16ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:34.966640 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd2c246-3ba0-463b-ac38-68d081aa16ad admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:34.979021 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1311/2636] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:29:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:29:34.983778 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46eb5ca-8f01-4be2-bbff-3ddc6734b629 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:34.994314 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46eb5ca-8f01-4be2-bbff-3ddc6734b629 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:35.000763 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee53b8a-1e3d-4f00-a962-e988092adfed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:35.001386 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee53b8a-1e3d-4f00-a962-e988092adfed admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:35.001686 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee53b8a-1e3d-4f00-a962-e988092adfed admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:35.001969 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee53b8a-1e3d-4f00-a962-e988092adfed admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:35.002650 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee53b8a-1e3d-4f00-a962-e988092adfed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:35.004587 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee53b8a-1e3d-4f00-a962-e988092adfed admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:35.041695 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1326/2637] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:29:34 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:29:35.043249 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd2c246-3ba0-463b-ac38-68d081aa16ad admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:35.053463 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd2c246-3ba0-463b-ac38-68d081aa16ad admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:35.083129 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:35.118526 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb6166d-0cb2-4080-a029-b85d981c08d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:35.119061 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb6166d-0cb2-4080-a029-b85d981c08d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:35.119448 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb6166d-0cb2-4080-a029-b85d981c08d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:35.119668 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb6166d-0cb2-4080-a029-b85d981c08d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:35.120331 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb6166d-0cb2-4080-a029-b85d981c08d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:35.122364 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb6166d-0cb2-4080-a029-b85d981c08d4 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:35.147632 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1312/2638] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:29:35 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:29:35.152771 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee53b8a-1e3d-4f00-a962-e988092adfed admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:35.164757 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee53b8a-1e3d-4f00-a962-e988092adfed admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:35.170640 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91921272-dde4-47dc-bc21-f211f4a0058f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:35.171258 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91921272-dde4-47dc-bc21-f211f4a0058f admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:35.171530 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91921272-dde4-47dc-bc21-f211f4a0058f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:35.171818 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91921272-dde4-47dc-bc21-f211f4a0058f admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:35.172475 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91921272-dde4-47dc-bc21-f211f4a0058f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:35.174135 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91921272-dde4-47dc-bc21-f211f4a0058f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:35.185772 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1327/2639] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:29:35.190345 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb6166d-0cb2-4080-a029-b85d981c08d4 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:35.200979 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb6166d-0cb2-4080-a029-b85d981c08d4 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:35.206225 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9f19af-1167-4d94-bb9d-24a743f30f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:35.206818 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f19af-1167-4d94-bb9d-24a743f30f60 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:35.207117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f19af-1167-4d94-bb9d-24a743f30f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:35.207379 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f19af-1167-4d94-bb9d-24a743f30f60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:35.208027 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9f19af-1167-4d94-bb9d-24a743f30f60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:35.209654 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9f19af-1167-4d94-bb9d-24a743f30f60 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:35.221209 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1313/2640] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:29:35.225607 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91921272-dde4-47dc-bc21-f211f4a0058f admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:35.236494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91921272-dde4-47dc-bc21-f211f4a0058f admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:35.241920 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb7bf1-fcca-4d13-8f43-01af00d389bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:35.242622 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb7bf1-fcca-4d13-8f43-01af00d389bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:35.242909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb7bf1-fcca-4d13-8f43-01af00d389bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:35.243168 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb7bf1-fcca-4d13-8f43-01af00d389bc admin admin] PATH_INFO: `/v3/projects/2058a4366c004f4386b972a834711a21/users/9705f8ad29384de7a70914444ee77e46/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:35.258866 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eb7bf1-fcca-4d13-8f43-01af00d389bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2058a4366c004f4386b972a834711a21, user_id=9705f8ad29384de7a70914444ee77e46, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:35.261643 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eb7bf1-fcca-4d13-8f43-01af00d389bc admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:35.277674 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1328/2641] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:29:35 2022] PUT /identity/v3/projects/2058a4366c004f4386b972a834711a21/users/9705f8ad29384de7a70914444ee77e46/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:35.294419 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb8c53-bff2-40bb-85ad-201130ed0953 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:35.294706 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb8c53-bff2-40bb-85ad-201130ed0953 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:35.294990 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb8c53-bff2-40bb-85ad-201130ed0953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:35.324440 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-e0eb8c53-bff2-40bb-85ad-201130ed0953 None None] MFA Rules not processed for user `9705f8ad29384de7a70914444ee77e46`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:29:35.325950 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-e0eb8c53-bff2-40bb-85ad-201130ed0953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:35.393915 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1314/2642] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:29:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:29:35.398514 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb7bf1-fcca-4d13-8f43-01af00d389bc admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:35.466206 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb7bf1-fcca-4d13-8f43-01af00d389bc admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:35.472150 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24805f6d-21e4-4132-8faf-abcbaa0c6575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:35.472757 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24805f6d-21e4-4132-8faf-abcbaa0c6575 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:35.473030 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24805f6d-21e4-4132-8faf-abcbaa0c6575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:35.473293 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24805f6d-21e4-4132-8faf-abcbaa0c6575 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:35.473919 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24805f6d-21e4-4132-8faf-abcbaa0c6575 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:35.475855 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24805f6d-21e4-4132-8faf-abcbaa0c6575 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:35.506448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1329/2643] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:29:35 2022] POST /identity/v3/projects => generated 387 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:29:35.510588 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eb8c53-bff2-40bb-85ad-201130ed0953 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:35.521728 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eb8c53-bff2-40bb-85ad-201130ed0953 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:35.585812 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d43be3a-07d3-4d74-a469-e33d0004a897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:35.586460 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d43be3a-07d3-4d74-a469-e33d0004a897 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:35.586734 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d43be3a-07d3-4d74-a469-e33d0004a897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:35.587018 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d43be3a-07d3-4d74-a469-e33d0004a897 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:35.587728 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d43be3a-07d3-4d74-a469-e33d0004a897 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:35.589719 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d43be3a-07d3-4d74-a469-e33d0004a897 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:35.616098 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1315/2644] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:29:35 2022] POST /identity/v3/users => generated 300 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:29:35.620724 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24805f6d-21e4-4132-8faf-abcbaa0c6575 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:35.632593 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24805f6d-21e4-4132-8faf-abcbaa0c6575 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:35.638476 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725dcd01-db99-4956-805a-ef8d80eb4ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:35.639145 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725dcd01-db99-4956-805a-ef8d80eb4ff3 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:35.639357 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725dcd01-db99-4956-805a-ef8d80eb4ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:35.639621 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725dcd01-db99-4956-805a-ef8d80eb4ff3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:35.640266 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725dcd01-db99-4956-805a-ef8d80eb4ff3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:35.641902 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725dcd01-db99-4956-805a-ef8d80eb4ff3 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:35.653210 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1330/2645] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 05:29:35.657654 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d43be3a-07d3-4d74-a469-e33d0004a897 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:35.668582 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d43be3a-07d3-4d74-a469-e33d0004a897 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:35.673680 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feaf49d-9379-46a6-939b-ad5665ae8d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:35.674401 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feaf49d-9379-46a6-939b-ad5665ae8d15 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:35.674678 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feaf49d-9379-46a6-939b-ad5665ae8d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:35.674971 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feaf49d-9379-46a6-939b-ad5665ae8d15 admin admin] PATH_INFO: `/v3/projects/98a2fe379e9a4a01b95d0e2e977b6021/users/91cd1d1b035c41059229e723624a9135/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:35.690769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feaf49d-9379-46a6-939b-ad5665ae8d15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98a2fe379e9a4a01b95d0e2e977b6021, user_id=91cd1d1b035c41059229e723624a9135, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:35.693448 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feaf49d-9379-46a6-939b-ad5665ae8d15 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:35.708861 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1316/2646] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:29:35 2022] PUT /identity/v3/projects/98a2fe379e9a4a01b95d0e2e977b6021/users/91cd1d1b035c41059229e723624a9135/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:35.713055 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725dcd01-db99-4956-805a-ef8d80eb4ff3 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:35.724429 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725dcd01-db99-4956-805a-ef8d80eb4ff3 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:35.747009 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:35.789149 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e5643-fe4b-4305-9664-032034d3e9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:35.789754 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e5643-fe4b-4305-9664-032034d3e9f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:35.790062 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e5643-fe4b-4305-9664-032034d3e9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:35.790349 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e5643-fe4b-4305-9664-032034d3e9f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:35.791016 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e5643-fe4b-4305-9664-032034d3e9f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:35.792729 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e5643-fe4b-4305-9664-032034d3e9f8 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:35.802861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1331/2647] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:29:35.807157 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feaf49d-9379-46a6-939b-ad5665ae8d15 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:35.812712 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:35.818631 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feaf49d-9379-46a6-939b-ad5665ae8d15 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:35.824946 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf3362-bebc-401d-be00-a4223923e139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:35.825606 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf3362-bebc-401d-be00-a4223923e139 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:35.825876 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf3362-bebc-401d-be00-a4223923e139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:35.826297 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf3362-bebc-401d-be00-a4223923e139 admin admin] PATH_INFO: `/v3/projects/98a2fe379e9a4a01b95d0e2e977b6021/users/91cd1d1b035c41059229e723624a9135/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:35.828310 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cf3362-bebc-401d-be00-a4223923e139 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98a2fe379e9a4a01b95d0e2e977b6021, user_id=91cd1d1b035c41059229e723624a9135, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:35.830986 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cf3362-bebc-401d-be00-a4223923e139 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:35.841719 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1317/2648] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:29:35 2022] PUT /identity/v3/projects/98a2fe379e9a4a01b95d0e2e977b6021/users/91cd1d1b035c41059229e723624a9135/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:35.852396 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4befffdd-08d6-44bb-bb79-a80b8bd83cf5 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:35.852710 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4befffdd-08d6-44bb-bb79-a80b8bd83cf5 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:35.852975 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4befffdd-08d6-44bb-bb79-a80b8bd83cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:35.883365 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-4befffdd-08d6-44bb-bb79-a80b8bd83cf5 None None] MFA Rules not processed for user `91cd1d1b035c41059229e723624a9135`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:29:35.884573 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4befffdd-08d6-44bb-bb79-a80b8bd83cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:35.954780 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1332/2649] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:29:35 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:29:35.966340 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf3362-bebc-401d-be00-a4223923e139 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:35.968340 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-24cf3362-bebc-401d-be00-a4223923e139 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:36.031961 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf3362-bebc-401d-be00-a4223923e139 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:36.035941 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ca0830-e39e-4d1b-a1ce-84f9f3fb586b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:36.036616 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca0830-e39e-4d1b-a1ce-84f9f3fb586b service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:36.036913 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca0830-e39e-4d1b-a1ce-84f9f3fb586b service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:36.037175 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca0830-e39e-4d1b-a1ce-84f9f3fb586b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:36.046781 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ca0830-e39e-4d1b-a1ce-84f9f3fb586b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:36.048240 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ca0830-e39e-4d1b-a1ce-84f9f3fb586b service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:36.058842 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1318/2650] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:29:35 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:29:36.087650 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4befffdd-08d6-44bb-bb79-a80b8bd83cf5 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:36.154270 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4befffdd-08d6-44bb-bb79-a80b8bd83cf5 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:36.159322 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a5525f-568e-4826-9531-46f7f693f9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:36.159947 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a5525f-568e-4826-9531-46f7f693f9db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:36.160223 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a5525f-568e-4826-9531-46f7f693f9db admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:36.160490 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a5525f-568e-4826-9531-46f7f693f9db admin admin] PATH_INFO: `/v3/users/9705f8ad29384de7a70914444ee77e46` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:36.161628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a5525f-568e-4826-9531-46f7f693f9db admin admin] RBAC: Authorizing `identity:delete_user(user_id=9705f8ad29384de7a70914444ee77e46)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:36.163573 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a5525f-568e-4826-9531-46f7f693f9db admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:36.218358 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-05a5525f-568e-4826-9531-46f7f693f9db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9705f8ad29384de7a70914444ee77e46'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:36.219767 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:36.228620 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-05a5525f-568e-4826-9531-46f7f693f9db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9705f8ad29384de7a70914444ee77e46'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:36.236635 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-05a5525f-568e-4826-9531-46f7f693f9db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9705f8ad29384de7a70914444ee77e46'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:36.237303 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-05a5525f-568e-4826-9531-46f7f693f9db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9705f8ad29384de7a70914444ee77e46'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:36.249728 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-05a5525f-568e-4826-9531-46f7f693f9db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9705f8ad29384de7a70914444ee77e46'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:36.256040 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1333/2651] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:29:36 2022] DELETE /identity/v3/users/9705f8ad29384de7a70914444ee77e46 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:29:36.261134 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222a14b-7dbb-4e7d-8ef6-b89bc7d66d2a None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:36.261394 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222a14b-7dbb-4e7d-8ef6-b89bc7d66d2a None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:36.261693 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222a14b-7dbb-4e7d-8ef6-b89bc7d66d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:36.282168 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-6222a14b-7dbb-4e7d-8ef6-b89bc7d66d2a None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:29:36.283419 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-6222a14b-7dbb-4e7d-8ef6-b89bc7d66d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:36.346582 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1319/2652] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:29:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:29:36.356988 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a5525f-568e-4826-9531-46f7f693f9db admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:36.358592 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-05a5525f-568e-4826-9531-46f7f693f9db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:36.417005 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a5525f-568e-4826-9531-46f7f693f9db admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:36.420895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f1b66-d4e3-4a9b-89d8-c33b4e43935c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:36.421516 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f1b66-d4e3-4a9b-89d8-c33b4e43935c service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:36.421789 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f1b66-d4e3-4a9b-89d8-c33b4e43935c service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:36.422050 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f1b66-d4e3-4a9b-89d8-c33b4e43935c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:36.435436 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8f1b66-d4e3-4a9b-89d8-c33b4e43935c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:36.436907 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8f1b66-d4e3-4a9b-89d8-c33b4e43935c service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:36.446849 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1334/2653] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:29:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:29:36.726696 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6222a14b-7dbb-4e7d-8ef6-b89bc7d66d2a None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:36.728586 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-6222a14b-7dbb-4e7d-8ef6-b89bc7d66d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:36.742657 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6222a14b-7dbb-4e7d-8ef6-b89bc7d66d2a None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:36.751665 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0517c3-5c07-45be-afad-f8e93bb41600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:36.752324 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0517c3-5c07-45be-afad-f8e93bb41600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:36.752600 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0517c3-5c07-45be-afad-f8e93bb41600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:36.752868 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0517c3-5c07-45be-afad-f8e93bb41600 admin admin] PATH_INFO: `/v3/projects/2058a4366c004f4386b972a834711a21` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:36.754380 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0517c3-5c07-45be-afad-f8e93bb41600 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2058a4366c004f4386b972a834711a21)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:36.756165 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0517c3-5c07-45be-afad-f8e93bb41600 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:36.806790 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0d0517c3-5c07-45be-afad-f8e93bb41600 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2058a4366c004f4386b972a834711a21', 'request_id': 'req-0d0517c3-5c07-45be-afad-f8e93bb41600'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:36.813334 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0d0517c3-5c07-45be-afad-f8e93bb41600 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2058a4366c004f4386b972a834711a21', 'request_id': 'req-0d0517c3-5c07-45be-afad-f8e93bb41600'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:36.819032 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0d0517c3-5c07-45be-afad-f8e93bb41600 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2058a4366c004f4386b972a834711a21', 'request_id': 'req-0d0517c3-5c07-45be-afad-f8e93bb41600'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:36.834363 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0d0517c3-5c07-45be-afad-f8e93bb41600 admin admin] The token cache is being invalidate because project 2058a4366c004f4386b972a834711a21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:29:36.834757 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0d0517c3-5c07-45be-afad-f8e93bb41600 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:36.835446 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0d0517c3-5c07-45be-afad-f8e93bb41600 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:36.837108 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1320/2654] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:29:36 2022] DELETE /identity/v3/projects/2058a4366c004f4386b972a834711a21 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:29:36.841327 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f1b66-d4e3-4a9b-89d8-c33b4e43935c service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:36.851969 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f1b66-d4e3-4a9b-89d8-c33b4e43935c service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:36.912283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7f7fe4-4ea3-4152-9d94-5cff640f4b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:36.912910 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f7fe4-4ea3-4152-9d94-5cff640f4b73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:36.913185 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f7fe4-4ea3-4152-9d94-5cff640f4b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:36.913452 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f7fe4-4ea3-4152-9d94-5cff640f4b73 admin admin] PATH_INFO: `/v3/users/91cd1d1b035c41059229e723624a9135` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:36.914618 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7f7fe4-4ea3-4152-9d94-5cff640f4b73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91cd1d1b035c41059229e723624a9135)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:36.916537 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7f7fe4-4ea3-4152-9d94-5cff640f4b73 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:36.967838 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8b7f7fe4-4ea3-4152-9d94-5cff640f4b73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91cd1d1b035c41059229e723624a9135'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:36.975670 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8b7f7fe4-4ea3-4152-9d94-5cff640f4b73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91cd1d1b035c41059229e723624a9135'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:36.986042 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8b7f7fe4-4ea3-4152-9d94-5cff640f4b73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91cd1d1b035c41059229e723624a9135'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:36.986742 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8b7f7fe4-4ea3-4152-9d94-5cff640f4b73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91cd1d1b035c41059229e723624a9135'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:36.992105 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:37.000623 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8b7f7fe4-4ea3-4152-9d94-5cff640f4b73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91cd1d1b035c41059229e723624a9135'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:37.007294 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1335/2655] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:29:36 2022] DELETE /identity/v3/users/91cd1d1b035c41059229e723624a9135 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:37.302158 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0517c3-5c07-45be-afad-f8e93bb41600 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:37.303603 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-0d0517c3-5c07-45be-afad-f8e93bb41600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:37.319236 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0517c3-5c07-45be-afad-f8e93bb41600 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:37.333461 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:37.386539 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c51319-58c0-4f77-9288-5b95b3d3a155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:37.387212 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c51319-58c0-4f77-9288-5b95b3d3a155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:37.387492 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c51319-58c0-4f77-9288-5b95b3d3a155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:37.387745 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c51319-58c0-4f77-9288-5b95b3d3a155 admin admin] PATH_INFO: `/v3/projects/98a2fe379e9a4a01b95d0e2e977b6021` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:37.388967 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c51319-58c0-4f77-9288-5b95b3d3a155 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98a2fe379e9a4a01b95d0e2e977b6021)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:37.390988 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c51319-58c0-4f77-9288-5b95b3d3a155 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:37.439612 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-68c51319-58c0-4f77-9288-5b95b3d3a155 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98a2fe379e9a4a01b95d0e2e977b6021', 'request_id': 'req-68c51319-58c0-4f77-9288-5b95b3d3a155'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:37.446442 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-68c51319-58c0-4f77-9288-5b95b3d3a155 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98a2fe379e9a4a01b95d0e2e977b6021', 'request_id': 'req-68c51319-58c0-4f77-9288-5b95b3d3a155'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:37.452675 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-68c51319-58c0-4f77-9288-5b95b3d3a155 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98a2fe379e9a4a01b95d0e2e977b6021', 'request_id': 'req-68c51319-58c0-4f77-9288-5b95b3d3a155'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:37.465918 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-68c51319-58c0-4f77-9288-5b95b3d3a155 admin admin] The token cache is being invalidate because project 98a2fe379e9a4a01b95d0e2e977b6021 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:29:37.466251 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-68c51319-58c0-4f77-9288-5b95b3d3a155 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:37.467080 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-68c51319-58c0-4f77-9288-5b95b3d3a155 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:37.472655 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1321/2656] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:29:37 2022] DELETE /identity/v3/projects/98a2fe379e9a4a01b95d0e2e977b6021 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:37.474272 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4ac72-fc6a-4e0f-83bb-013c7fe8f5ae None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:37.474567 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4ac72-fc6a-4e0f-83bb-013c7fe8f5ae None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:37.474840 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4ac72-fc6a-4e0f-83bb-013c7fe8f5ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:37.495416 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-15f4ac72-fc6a-4e0f-83bb-013c7fe8f5ae None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:29:37.496633 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-15f4ac72-fc6a-4e0f-83bb-013c7fe8f5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:37.557116 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1336/2657] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:29:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:29:37.565255 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa9b1a-eabc-4f10-a8f6-9711b777e3e6 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:37.565541 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa9b1a-eabc-4f10-a8f6-9711b777e3e6 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:37.565810 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa9b1a-eabc-4f10-a8f6-9711b777e3e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:37.590846 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-2dfa9b1a-eabc-4f10-a8f6-9711b777e3e6 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:29:37.592113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-2dfa9b1a-eabc-4f10-a8f6-9711b777e3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:37.602622 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1322/2658] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:29:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:29:37.607939 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f4ac72-fc6a-4e0f-83bb-013c7fe8f5ae None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:37.621861 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f4ac72-fc6a-4e0f-83bb-013c7fe8f5ae None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:37.627919 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffe564f-0831-4af0-9901-8ce394720356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:37.628548 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe564f-0831-4af0-9901-8ce394720356 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:37.628822 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe564f-0831-4af0-9901-8ce394720356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:37.629120 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe564f-0831-4af0-9901-8ce394720356 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:37.629779 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffe564f-0831-4af0-9901-8ce394720356 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:29:37.630129 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffe564f-0831-4af0-9901-8ce394720356 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:37.631767 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffe564f-0831-4af0-9901-8ce394720356 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:37.645231 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1337/2659] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:29:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:29:37.649710 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfa9b1a-eabc-4f10-a8f6-9711b777e3e6 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:37.660724 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfa9b1a-eabc-4f10-a8f6-9711b777e3e6 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:37.665813 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a9385c-5a61-4ff6-880a-75cf3f577693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:37.666473 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9385c-5a61-4ff6-880a-75cf3f577693 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:37.666751 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9385c-5a61-4ff6-880a-75cf3f577693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:37.667023 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9385c-5a61-4ff6-880a-75cf3f577693 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:37.667706 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a9385c-5a61-4ff6-880a-75cf3f577693 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:37.669647 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a9385c-5a61-4ff6-880a-75cf3f577693 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:37.701427 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1323/2660] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:29:37 2022] POST /identity/v3/projects => generated 387 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:29:37.705963 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffe564f-0831-4af0-9901-8ce394720356 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:37.716586 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffe564f-0831-4af0-9901-8ce394720356 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:37.777755 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a74044-5ef4-4adb-a987-74179f4db90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:37.778440 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a74044-5ef4-4adb-a987-74179f4db90e admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:37.778722 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a74044-5ef4-4adb-a987-74179f4db90e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:37.778995 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a74044-5ef4-4adb-a987-74179f4db90e admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:37.779641 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a74044-5ef4-4adb-a987-74179f4db90e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:37.781571 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a74044-5ef4-4adb-a987-74179f4db90e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:37.806211 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1338/2661] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:29:37 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:29:37.810615 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a9385c-5a61-4ff6-880a-75cf3f577693 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:37.821293 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a9385c-5a61-4ff6-880a-75cf3f577693 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:37.826156 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5001b56b-e8ba-4e89-8046-5db25d5c2e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:37.826766 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001b56b-e8ba-4e89-8046-5db25d5c2e4c admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:37.827056 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001b56b-e8ba-4e89-8046-5db25d5c2e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:37.827332 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001b56b-e8ba-4e89-8046-5db25d5c2e4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:37.827990 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5001b56b-e8ba-4e89-8046-5db25d5c2e4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:37.829649 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5001b56b-e8ba-4e89-8046-5db25d5c2e4c admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:37.841929 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1324/2662] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:29:37.846683 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a74044-5ef4-4adb-a987-74179f4db90e admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:37.858241 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a74044-5ef4-4adb-a987-74179f4db90e admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:37.863482 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e47eb2c-0d27-471f-bae4-9d9741ec2d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:37.864083 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47eb2c-0d27-471f-bae4-9d9741ec2d42 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:37.864363 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47eb2c-0d27-471f-bae4-9d9741ec2d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:37.864616 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47eb2c-0d27-471f-bae4-9d9741ec2d42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:37.865274 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e47eb2c-0d27-471f-bae4-9d9741ec2d42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:37.866981 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e47eb2c-0d27-471f-bae4-9d9741ec2d42 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:37.877747 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1339/2663] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:29:37.882233 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5001b56b-e8ba-4e89-8046-5db25d5c2e4c admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:37.892988 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5001b56b-e8ba-4e89-8046-5db25d5c2e4c admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:37.898327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3615e926-8e54-4826-818b-35e9d5b54bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:37.899007 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3615e926-8e54-4826-818b-35e9d5b54bc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:37.899286 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3615e926-8e54-4826-818b-35e9d5b54bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:37.899558 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3615e926-8e54-4826-818b-35e9d5b54bc9 admin admin] PATH_INFO: `/v3/projects/9797bf5d4ddb460ebd9148e99546a0cf/users/9cbe41a64ead4151a0bd30784aad526c/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:37.918432 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3615e926-8e54-4826-818b-35e9d5b54bc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9797bf5d4ddb460ebd9148e99546a0cf, user_id=9cbe41a64ead4151a0bd30784aad526c, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:37.921184 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3615e926-8e54-4826-818b-35e9d5b54bc9 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:37.936946 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1325/2664] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:29:37 2022] PUT /identity/v3/projects/9797bf5d4ddb460ebd9148e99546a0cf/users/9cbe41a64ead4151a0bd30784aad526c/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:37.948453 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d23483-d84d-4f2f-acf5-ddaec89a5e2c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:37.948770 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d23483-d84d-4f2f-acf5-ddaec89a5e2c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:37.949041 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d23483-d84d-4f2f-acf5-ddaec89a5e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:37.978100 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-38d23483-d84d-4f2f-acf5-ddaec89a5e2c None None] MFA Rules not processed for user `9cbe41a64ead4151a0bd30784aad526c`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:29:37.979322 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-38d23483-d84d-4f2f-acf5-ddaec89a5e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:38.025021 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:38.054123 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1340/2665] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:29:37 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:29:38.058868 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3615e926-8e54-4826-818b-35e9d5b54bc9 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:38.108001 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:38.127967 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3615e926-8e54-4826-818b-35e9d5b54bc9 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:38.132574 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe86a37-6fea-4ce2-b963-a4fad84fc5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:38.133221 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe86a37-6fea-4ce2-b963-a4fad84fc5f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:38.133506 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe86a37-6fea-4ce2-b963-a4fad84fc5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:38.133756 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe86a37-6fea-4ce2-b963-a4fad84fc5f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:38.134571 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe86a37-6fea-4ce2-b963-a4fad84fc5f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:38.136682 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe86a37-6fea-4ce2-b963-a4fad84fc5f2 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:38.168694 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1326/2666] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:29:38 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:29:38.173149 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d23483-d84d-4f2f-acf5-ddaec89a5e2c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:38.184360 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d23483-d84d-4f2f-acf5-ddaec89a5e2c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:38.257282 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ad8dc-1fa1-4d27-adec-da15f07727c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:38.258127 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ad8dc-1fa1-4d27-adec-da15f07727c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:38.258517 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ad8dc-1fa1-4d27-adec-da15f07727c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:38.258901 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ad8dc-1fa1-4d27-adec-da15f07727c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:38.259970 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8ad8dc-1fa1-4d27-adec-da15f07727c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:38.262491 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8ad8dc-1fa1-4d27-adec-da15f07727c7 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:38.287318 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1341/2667] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:29:38 2022] POST /identity/v3/users => generated 299 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:29:38.291638 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe86a37-6fea-4ce2-b963-a4fad84fc5f2 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:38.302838 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe86a37-6fea-4ce2-b963-a4fad84fc5f2 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:38.308117 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b84ea-5560-4882-906a-b348b29e5d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:38.308645 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b84ea-5560-4882-906a-b348b29e5d46 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:38.308919 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b84ea-5560-4882-906a-b348b29e5d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:38.309182 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b84ea-5560-4882-906a-b348b29e5d46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:38.310023 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1b84ea-5560-4882-906a-b348b29e5d46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:38.311631 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1b84ea-5560-4882-906a-b348b29e5d46 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:38.322498 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1327/2668] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:29:38.327621 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ad8dc-1fa1-4d27-adec-da15f07727c7 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:38.342803 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ad8dc-1fa1-4d27-adec-da15f07727c7 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:38.350295 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acb2996-4be8-40be-ac8a-ab547f7c69e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:38.351128 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb2996-4be8-40be-ac8a-ab547f7c69e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:38.351418 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb2996-4be8-40be-ac8a-ab547f7c69e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:38.351806 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb2996-4be8-40be-ac8a-ab547f7c69e7 admin admin] PATH_INFO: `/v3/projects/04726c53a1a4463b819d966cfd755635/users/65179f233c3f4b7496ad196515289670/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:38.369133 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acb2996-4be8-40be-ac8a-ab547f7c69e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04726c53a1a4463b819d966cfd755635, user_id=65179f233c3f4b7496ad196515289670, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:38.371932 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acb2996-4be8-40be-ac8a-ab547f7c69e7 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:38.389072 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1342/2669] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:29:38 2022] PUT /identity/v3/projects/04726c53a1a4463b819d966cfd755635/users/65179f233c3f4b7496ad196515289670/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:38.394176 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b84ea-5560-4882-906a-b348b29e5d46 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:38.406387 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b84ea-5560-4882-906a-b348b29e5d46 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:38.466899 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a56815-7175-4760-a102-5f2fe7780811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:38.467829 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a56815-7175-4760-a102-5f2fe7780811 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:38.468328 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a56815-7175-4760-a102-5f2fe7780811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:38.468760 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a56815-7175-4760-a102-5f2fe7780811 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:38.469973 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a56815-7175-4760-a102-5f2fe7780811 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:38.471848 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a56815-7175-4760-a102-5f2fe7780811 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:38.482740 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1328/2670] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:29:38.487544 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acb2996-4be8-40be-ac8a-ab547f7c69e7 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:38.498432 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acb2996-4be8-40be-ac8a-ab547f7c69e7 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:38.503670 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959dc56c-4dc0-483f-831d-75df81c09e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:38.504459 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959dc56c-4dc0-483f-831d-75df81c09e0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:38.504733 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959dc56c-4dc0-483f-831d-75df81c09e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:38.505036 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959dc56c-4dc0-483f-831d-75df81c09e0c admin admin] PATH_INFO: `/v3/projects/04726c53a1a4463b819d966cfd755635/users/65179f233c3f4b7496ad196515289670/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:38.507206 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959dc56c-4dc0-483f-831d-75df81c09e0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=04726c53a1a4463b819d966cfd755635, user_id=65179f233c3f4b7496ad196515289670, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:38.509932 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959dc56c-4dc0-483f-831d-75df81c09e0c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:38.521315 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1343/2671] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:29:38 2022] PUT /identity/v3/projects/04726c53a1a4463b819d966cfd755635/users/65179f233c3f4b7496ad196515289670/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:29:38.532489 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dca478-81fb-4b10-bad8-7db718804ff9 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:38.532746 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dca478-81fb-4b10-bad8-7db718804ff9 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:38.533038 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dca478-81fb-4b10-bad8-7db718804ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:38.562960 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-42dca478-81fb-4b10-bad8-7db718804ff9 None None] MFA Rules not processed for user `65179f233c3f4b7496ad196515289670`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:29:38.564088 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-42dca478-81fb-4b10-bad8-7db718804ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:38.635175 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1329/2672] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:29:38 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:29:38.647459 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959dc56c-4dc0-483f-831d-75df81c09e0c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:38.650231 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-959dc56c-4dc0-483f-831d-75df81c09e0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:38.713881 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959dc56c-4dc0-483f-831d-75df81c09e0c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:38.718091 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ccd367-dab4-49db-9fe1-57572c10c8b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:38.718744 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ccd367-dab4-49db-9fe1-57572c10c8b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:38.719362 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ccd367-dab4-49db-9fe1-57572c10c8b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:38.719664 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ccd367-dab4-49db-9fe1-57572c10c8b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:38.729833 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ccd367-dab4-49db-9fe1-57572c10c8b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:38.731320 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ccd367-dab4-49db-9fe1-57572c10c8b6 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:38.740960 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1344/2673] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:29:38 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:29:39.347997 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dca478-81fb-4b10-bad8-7db718804ff9 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:39.389439 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:39.411417 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dca478-81fb-4b10-bad8-7db718804ff9 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:39.416139 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda60d97-aeb0-444a-9423-cac8eaa5fe09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:39.416768 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda60d97-aeb0-444a-9423-cac8eaa5fe09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:39.417040 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda60d97-aeb0-444a-9423-cac8eaa5fe09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:39.417318 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda60d97-aeb0-444a-9423-cac8eaa5fe09 admin admin] PATH_INFO: `/v3/users/9cbe41a64ead4151a0bd30784aad526c` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:39.418512 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda60d97-aeb0-444a-9423-cac8eaa5fe09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cbe41a64ead4151a0bd30784aad526c)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:39.420388 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda60d97-aeb0-444a-9423-cac8eaa5fe09 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:39.470542 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-dda60d97-aeb0-444a-9423-cac8eaa5fe09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cbe41a64ead4151a0bd30784aad526c'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:39.478808 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-dda60d97-aeb0-444a-9423-cac8eaa5fe09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cbe41a64ead4151a0bd30784aad526c'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:39.479623 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-dda60d97-aeb0-444a-9423-cac8eaa5fe09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cbe41a64ead4151a0bd30784aad526c'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:39.489635 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-dda60d97-aeb0-444a-9423-cac8eaa5fe09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cbe41a64ead4151a0bd30784aad526c'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:39.490491 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-dda60d97-aeb0-444a-9423-cac8eaa5fe09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cbe41a64ead4151a0bd30784aad526c'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:39.509824 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1330/2674] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:29:39 2022] DELETE /identity/v3/users/9cbe41a64ead4151a0bd30784aad526c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:39.514816 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7036047-00eb-45b8-afbe-7e47bb170b59 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:39.515360 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7036047-00eb-45b8-afbe-7e47bb170b59 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:39.515360 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7036047-00eb-45b8-afbe-7e47bb170b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:39.536306 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e7036047-00eb-45b8-afbe-7e47bb170b59 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:29:39.537560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e7036047-00eb-45b8-afbe-7e47bb170b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:39.572959 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:39.603129 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1345/2675] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:29:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:29:39.617805 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda60d97-aeb0-444a-9423-cac8eaa5fe09 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:39.620287 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-dda60d97-aeb0-444a-9423-cac8eaa5fe09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:39.693748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda60d97-aeb0-444a-9423-cac8eaa5fe09 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:39.698189 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f96c4e-f1e2-49e0-9315-5c6125502f4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:39.698873 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f96c4e-f1e2-49e0-9315-5c6125502f4e service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:39.699152 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f96c4e-f1e2-49e0-9315-5c6125502f4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:39.699400 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f96c4e-f1e2-49e0-9315-5c6125502f4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:39.712066 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f96c4e-f1e2-49e0-9315-5c6125502f4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:39.713565 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f96c4e-f1e2-49e0-9315-5c6125502f4e service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:39.726776 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1331/2676] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:29:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:29:40.021534 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7036047-00eb-45b8-afbe-7e47bb170b59 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:40.023238 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e7036047-00eb-45b8-afbe-7e47bb170b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:40.036009 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7036047-00eb-45b8-afbe-7e47bb170b59 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:40.041189 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d92acc2-0ee8-4091-a75a-a3b9ebd3b843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:40.041832 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92acc2-0ee8-4091-a75a-a3b9ebd3b843 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:40.042139 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92acc2-0ee8-4091-a75a-a3b9ebd3b843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:40.042410 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92acc2-0ee8-4091-a75a-a3b9ebd3b843 admin admin] PATH_INFO: `/v3/projects/9797bf5d4ddb460ebd9148e99546a0cf` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:40.043542 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d92acc2-0ee8-4091-a75a-a3b9ebd3b843 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9797bf5d4ddb460ebd9148e99546a0cf)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:40.045466 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d92acc2-0ee8-4091-a75a-a3b9ebd3b843 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:40.097069 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7d92acc2-0ee8-4091-a75a-a3b9ebd3b843 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9797bf5d4ddb460ebd9148e99546a0cf', 'request_id': 'req-7d92acc2-0ee8-4091-a75a-a3b9ebd3b843'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:40.103358 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7d92acc2-0ee8-4091-a75a-a3b9ebd3b843 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9797bf5d4ddb460ebd9148e99546a0cf', 'request_id': 'req-7d92acc2-0ee8-4091-a75a-a3b9ebd3b843'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:40.113397 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7d92acc2-0ee8-4091-a75a-a3b9ebd3b843 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9797bf5d4ddb460ebd9148e99546a0cf', 'request_id': 'req-7d92acc2-0ee8-4091-a75a-a3b9ebd3b843'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:40.126528 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7d92acc2-0ee8-4091-a75a-a3b9ebd3b843 admin admin] The token cache is being invalidate because project 9797bf5d4ddb460ebd9148e99546a0cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:29:40.126782 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7d92acc2-0ee8-4091-a75a-a3b9ebd3b843 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:40.129063 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7d92acc2-0ee8-4091-a75a-a3b9ebd3b843 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:40.130623 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1346/2677] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:29:40 2022] DELETE /identity/v3/projects/9797bf5d4ddb460ebd9148e99546a0cf => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:40.135027 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f96c4e-f1e2-49e0-9315-5c6125502f4e service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:40.145526 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f96c4e-f1e2-49e0-9315-5c6125502f4e service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:40.204667 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf4f41d-4228-4f5d-bb6a-6e012675e22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:40.205313 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4f41d-4228-4f5d-bb6a-6e012675e22d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:40.205591 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4f41d-4228-4f5d-bb6a-6e012675e22d admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:40.205859 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4f41d-4228-4f5d-bb6a-6e012675e22d admin admin] PATH_INFO: `/v3/users/65179f233c3f4b7496ad196515289670` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:40.207060 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf4f41d-4228-4f5d-bb6a-6e012675e22d admin admin] RBAC: Authorizing `identity:delete_user(user_id=65179f233c3f4b7496ad196515289670)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:40.209137 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf4f41d-4228-4f5d-bb6a-6e012675e22d admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:40.210720 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:40.271369 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-2cf4f41d-4228-4f5d-bb6a-6e012675e22d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65179f233c3f4b7496ad196515289670'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:40.273715 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-2cf4f41d-4228-4f5d-bb6a-6e012675e22d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65179f233c3f4b7496ad196515289670'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:40.274429 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-2cf4f41d-4228-4f5d-bb6a-6e012675e22d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65179f233c3f4b7496ad196515289670'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:40.283516 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-2cf4f41d-4228-4f5d-bb6a-6e012675e22d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65179f233c3f4b7496ad196515289670'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:40.284236 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-2cf4f41d-4228-4f5d-bb6a-6e012675e22d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65179f233c3f4b7496ad196515289670'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:40.302229 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1332/2678] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:29:40 2022] DELETE /identity/v3/users/65179f233c3f4b7496ad196515289670 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:40.592094 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d92acc2-0ee8-4091-a75a-a3b9ebd3b843 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:40.593754 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7d92acc2-0ee8-4091-a75a-a3b9ebd3b843 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:40.609260 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d92acc2-0ee8-4091-a75a-a3b9ebd3b843 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:40.671932 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319aed8-c8e6-4b0b-aa96-642ad49d6d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:40.672605 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319aed8-c8e6-4b0b-aa96-642ad49d6d91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:40.672879 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319aed8-c8e6-4b0b-aa96-642ad49d6d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:40.673144 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319aed8-c8e6-4b0b-aa96-642ad49d6d91 admin admin] PATH_INFO: `/v3/projects/04726c53a1a4463b819d966cfd755635` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:40.674291 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3319aed8-c8e6-4b0b-aa96-642ad49d6d91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04726c53a1a4463b819d966cfd755635)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:40.676211 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3319aed8-c8e6-4b0b-aa96-642ad49d6d91 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:40.728493 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3319aed8-c8e6-4b0b-aa96-642ad49d6d91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04726c53a1a4463b819d966cfd755635', 'request_id': 'req-3319aed8-c8e6-4b0b-aa96-642ad49d6d91'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:40.735518 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3319aed8-c8e6-4b0b-aa96-642ad49d6d91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04726c53a1a4463b819d966cfd755635', 'request_id': 'req-3319aed8-c8e6-4b0b-aa96-642ad49d6d91'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:40.745214 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3319aed8-c8e6-4b0b-aa96-642ad49d6d91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04726c53a1a4463b819d966cfd755635', 'request_id': 'req-3319aed8-c8e6-4b0b-aa96-642ad49d6d91'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:40.748554 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:40.760374 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3319aed8-c8e6-4b0b-aa96-642ad49d6d91 admin admin] The token cache is being invalidate because project 04726c53a1a4463b819d966cfd755635 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:29:40.760839 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3319aed8-c8e6-4b0b-aa96-642ad49d6d91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:40.761518 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3319aed8-c8e6-4b0b-aa96-642ad49d6d91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:40.763106 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1347/2679] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:29:40 2022] DELETE /identity/v3/projects/04726c53a1a4463b819d966cfd755635 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:40.768458 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28864438-fd29-4646-af40-39c4c6c98d6e None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:40.768926 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28864438-fd29-4646-af40-39c4c6c98d6e None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:40.769023 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28864438-fd29-4646-af40-39c4c6c98d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:40.790520 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-28864438-fd29-4646-af40-39c4c6c98d6e None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:29:40.791791 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-28864438-fd29-4646-af40-39c4c6c98d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:40.854240 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1333/2680] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:29:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:29:40.862452 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd97466-e983-471d-b13c-0928f9fb4850 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:40.862746 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd97466-e983-471d-b13c-0928f9fb4850 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:40.863039 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd97466-e983-471d-b13c-0928f9fb4850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:40.884820 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-afd97466-e983-471d-b13c-0928f9fb4850 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:29:40.886043 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-afd97466-e983-471d-b13c-0928f9fb4850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:40.895288 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1348/2681] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:29:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:29:40.899882 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28864438-fd29-4646-af40-39c4c6c98d6e None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:40.910196 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28864438-fd29-4646-af40-39c4c6c98d6e None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:40.915739 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af677b-ec9b-4282-9035-6e63d1bdfc8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:40.916368 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af677b-ec9b-4282-9035-6e63d1bdfc8f admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:40.916633 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af677b-ec9b-4282-9035-6e63d1bdfc8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:40.917005 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af677b-ec9b-4282-9035-6e63d1bdfc8f admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:40.917667 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12af677b-ec9b-4282-9035-6e63d1bdfc8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:29:40.917968 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12af677b-ec9b-4282-9035-6e63d1bdfc8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:40.919644 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12af677b-ec9b-4282-9035-6e63d1bdfc8f admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:40.932278 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1334/2682] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:29:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:29:40.937059 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd97466-e983-471d-b13c-0928f9fb4850 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:40.946894 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd97466-e983-471d-b13c-0928f9fb4850 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:40.952190 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356a436a-c84f-4512-9db0-a2f996bfb109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:40.952786 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a436a-c84f-4512-9db0-a2f996bfb109 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:40.953060 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a436a-c84f-4512-9db0-a2f996bfb109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:40.953329 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a436a-c84f-4512-9db0-a2f996bfb109 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:40.953963 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356a436a-c84f-4512-9db0-a2f996bfb109 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:40.955902 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356a436a-c84f-4512-9db0-a2f996bfb109 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:40.985961 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1349/2683] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:29:40 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:29:40.990892 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af677b-ec9b-4282-9035-6e63d1bdfc8f admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:41.003240 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af677b-ec9b-4282-9035-6e63d1bdfc8f admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:41.060416 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c7a05-7621-4eae-a7fd-cbde39903d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:41.060950 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c7a05-7621-4eae-a7fd-cbde39903d8f admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:41.061226 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c7a05-7621-4eae-a7fd-cbde39903d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:41.061493 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c7a05-7621-4eae-a7fd-cbde39903d8f admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:41.062175 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c7a05-7621-4eae-a7fd-cbde39903d8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:41.064110 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c7a05-7621-4eae-a7fd-cbde39903d8f admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:41.075340 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:41.088333 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1335/2684] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:29:40 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:29:41.092565 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356a436a-c84f-4512-9db0-a2f996bfb109 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:41.103242 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356a436a-c84f-4512-9db0-a2f996bfb109 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:41.108199 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90e000-fa6c-4d83-9be3-01a6f0cba367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:41.108825 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90e000-fa6c-4d83-9be3-01a6f0cba367 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:41.109114 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90e000-fa6c-4d83-9be3-01a6f0cba367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:41.109379 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90e000-fa6c-4d83-9be3-01a6f0cba367 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:41.110046 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae90e000-fa6c-4d83-9be3-01a6f0cba367 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:41.111643 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae90e000-fa6c-4d83-9be3-01a6f0cba367 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:41.122044 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1350/2685] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:29:41.126752 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c7a05-7621-4eae-a7fd-cbde39903d8f admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:41.137615 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c7a05-7621-4eae-a7fd-cbde39903d8f admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:41.143452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2579b26-87be-4ed6-b324-38b2244113bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:41.144193 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2579b26-87be-4ed6-b324-38b2244113bb admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:41.144450 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2579b26-87be-4ed6-b324-38b2244113bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:41.144728 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2579b26-87be-4ed6-b324-38b2244113bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:41.145371 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2579b26-87be-4ed6-b324-38b2244113bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:41.147045 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2579b26-87be-4ed6-b324-38b2244113bb admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:41.158265 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1336/2686] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:29:41.162860 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90e000-fa6c-4d83-9be3-01a6f0cba367 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:41.173315 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90e000-fa6c-4d83-9be3-01a6f0cba367 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:41.178615 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2275d24-45de-4dca-9447-3a25af58977d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:41.179283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2275d24-45de-4dca-9447-3a25af58977d admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:41.179560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2275d24-45de-4dca-9447-3a25af58977d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:41.179830 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2275d24-45de-4dca-9447-3a25af58977d admin admin] PATH_INFO: `/v3/projects/2ab062e3d79a48bc8cf071568fb60329/users/1c371a7a764c4dc19c8d481f6119684c/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:41.197025 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2275d24-45de-4dca-9447-3a25af58977d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ab062e3d79a48bc8cf071568fb60329, user_id=1c371a7a764c4dc19c8d481f6119684c, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:41.199773 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2275d24-45de-4dca-9447-3a25af58977d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:41.216060 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1351/2687] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:29:41 2022] PUT /identity/v3/projects/2ab062e3d79a48bc8cf071568fb60329/users/1c371a7a764c4dc19c8d481f6119684c/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:41.232661 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606de04a-c496-4371-8218-7b09de220282 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:41.232941 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606de04a-c496-4371-8218-7b09de220282 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:41.233204 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606de04a-c496-4371-8218-7b09de220282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:41.261042 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-606de04a-c496-4371-8218-7b09de220282 None None] MFA Rules not processed for user `1c371a7a764c4dc19c8d481f6119684c`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:29:41.262405 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-606de04a-c496-4371-8218-7b09de220282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:41.330176 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1337/2688] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:29:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:29:41.335604 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2275d24-45de-4dca-9447-3a25af58977d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:41.534871 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2275d24-45de-4dca-9447-3a25af58977d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:41.540906 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87041f19-50ee-476b-8a16-6b193204d8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:41.541521 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87041f19-50ee-476b-8a16-6b193204d8eb admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:41.541801 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87041f19-50ee-476b-8a16-6b193204d8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:41.542083 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87041f19-50ee-476b-8a16-6b193204d8eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:41.542709 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87041f19-50ee-476b-8a16-6b193204d8eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:41.544904 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87041f19-50ee-476b-8a16-6b193204d8eb admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:41.574269 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1352/2689] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:29:41 2022] POST /identity/v3/projects => generated 381 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:29:41.578574 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606de04a-c496-4371-8218-7b09de220282 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:41.589510 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606de04a-c496-4371-8218-7b09de220282 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:41.653265 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b92247-4090-4168-9607-82b704f28829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:41.653882 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b92247-4090-4168-9607-82b704f28829 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:41.654194 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b92247-4090-4168-9607-82b704f28829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:41.654459 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b92247-4090-4168-9607-82b704f28829 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:41.655092 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b92247-4090-4168-9607-82b704f28829 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:41.657066 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b92247-4090-4168-9607-82b704f28829 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:41.680206 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1338/2690] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:29:41 2022] POST /identity/v3/users => generated 297 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:29:41.684197 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87041f19-50ee-476b-8a16-6b193204d8eb admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:41.695173 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87041f19-50ee-476b-8a16-6b193204d8eb admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:41.701131 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b6f8b-873d-4ff3-b0d8-291fc10869e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:41.701734 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b6f8b-873d-4ff3-b0d8-291fc10869e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:41.702047 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b6f8b-873d-4ff3-b0d8-291fc10869e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:41.702311 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b6f8b-873d-4ff3-b0d8-291fc10869e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:41.702959 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187b6f8b-873d-4ff3-b0d8-291fc10869e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:41.704558 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187b6f8b-873d-4ff3-b0d8-291fc10869e5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:41.715569 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1353/2691] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:29:41.720232 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b92247-4090-4168-9607-82b704f28829 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:41.731532 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b92247-4090-4168-9607-82b704f28829 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:41.737494 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82cefa0-0284-4503-91a5-b0e92cadf726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:41.738185 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82cefa0-0284-4503-91a5-b0e92cadf726 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:41.738470 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82cefa0-0284-4503-91a5-b0e92cadf726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:41.738737 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82cefa0-0284-4503-91a5-b0e92cadf726 admin admin] PATH_INFO: `/v3/projects/23a14c9aa55949959f6c07883ed02ba8/users/cb59e6aecf214cd5a23202bc940097cc/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:41.755562 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82cefa0-0284-4503-91a5-b0e92cadf726 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23a14c9aa55949959f6c07883ed02ba8, user_id=cb59e6aecf214cd5a23202bc940097cc, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:41.758316 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82cefa0-0284-4503-91a5-b0e92cadf726 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:41.772831 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1339/2692] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:29:41 2022] PUT /identity/v3/projects/23a14c9aa55949959f6c07883ed02ba8/users/cb59e6aecf214cd5a23202bc940097cc/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:41.777035 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b6f8b-873d-4ff3-b0d8-291fc10869e5 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:41.787449 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b6f8b-873d-4ff3-b0d8-291fc10869e5 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:41.848439 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1638a4c3-a193-413d-a919-5441009d399b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:41.849190 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1638a4c3-a193-413d-a919-5441009d399b admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:41.849494 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1638a4c3-a193-413d-a919-5441009d399b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:41.849745 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1638a4c3-a193-413d-a919-5441009d399b admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:41.850414 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1638a4c3-a193-413d-a919-5441009d399b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:41.852001 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1638a4c3-a193-413d-a919-5441009d399b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:41.854148 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:41.863633 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1354/2693] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:29:41.868181 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82cefa0-0284-4503-91a5-b0e92cadf726 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:41.878781 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82cefa0-0284-4503-91a5-b0e92cadf726 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:41.883671 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c35e7-7388-4017-bb85-e85fff57d0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:41.884321 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c35e7-7388-4017-bb85-e85fff57d0e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:41.884600 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c35e7-7388-4017-bb85-e85fff57d0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:41.884871 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c35e7-7388-4017-bb85-e85fff57d0e8 admin admin] PATH_INFO: `/v3/projects/23a14c9aa55949959f6c07883ed02ba8/users/cb59e6aecf214cd5a23202bc940097cc/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:41.886989 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3c35e7-7388-4017-bb85-e85fff57d0e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23a14c9aa55949959f6c07883ed02ba8, user_id=cb59e6aecf214cd5a23202bc940097cc, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:41.889656 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3c35e7-7388-4017-bb85-e85fff57d0e8 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:41.900135 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1340/2694] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:29:41 2022] PUT /identity/v3/projects/23a14c9aa55949959f6c07883ed02ba8/users/cb59e6aecf214cd5a23202bc940097cc/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:41.910357 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5277744f-744e-45a5-906e-88234e9eb4ee None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:41.910644 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5277744f-744e-45a5-906e-88234e9eb4ee None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:41.910910 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5277744f-744e-45a5-906e-88234e9eb4ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:41.939810 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-5277744f-744e-45a5-906e-88234e9eb4ee None None] MFA Rules not processed for user `cb59e6aecf214cd5a23202bc940097cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:29:41.941034 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5277744f-744e-45a5-906e-88234e9eb4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:42.017315 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1355/2695] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:29:41 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:29:42.028304 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c35e7-7388-4017-bb85-e85fff57d0e8 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:42.030408 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-cf3c35e7-7388-4017-bb85-e85fff57d0e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:42.063185 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:42.093802 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c35e7-7388-4017-bb85-e85fff57d0e8 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:42.098869 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9366c3a-012e-45a0-b61e-beb6c3d2aca9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:42.099446 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9366c3a-012e-45a0-b61e-beb6c3d2aca9 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:42.099762 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9366c3a-012e-45a0-b61e-beb6c3d2aca9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:42.100077 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9366c3a-012e-45a0-b61e-beb6c3d2aca9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:42.111035 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9366c3a-012e-45a0-b61e-beb6c3d2aca9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:42.112494 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9366c3a-012e-45a0-b61e-beb6c3d2aca9 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:42.122379 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1341/2696] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:29:42 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:29:42.508332 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5277744f-744e-45a5-906e-88234e9eb4ee None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:42.571341 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5277744f-744e-45a5-906e-88234e9eb4ee None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:42.575993 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e019ed3-d0cf-439a-8a44-34d641546b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:42.576622 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e019ed3-d0cf-439a-8a44-34d641546b7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:42.576904 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e019ed3-d0cf-439a-8a44-34d641546b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:42.577165 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e019ed3-d0cf-439a-8a44-34d641546b7f admin admin] PATH_INFO: `/v3/users/1c371a7a764c4dc19c8d481f6119684c` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:42.578359 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e019ed3-d0cf-439a-8a44-34d641546b7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c371a7a764c4dc19c8d481f6119684c)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:42.580272 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e019ed3-d0cf-439a-8a44-34d641546b7f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:42.628890 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6e019ed3-d0cf-439a-8a44-34d641546b7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c371a7a764c4dc19c8d481f6119684c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:42.635578 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6e019ed3-d0cf-439a-8a44-34d641546b7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c371a7a764c4dc19c8d481f6119684c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:42.643556 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6e019ed3-d0cf-439a-8a44-34d641546b7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c371a7a764c4dc19c8d481f6119684c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:42.644194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6e019ed3-d0cf-439a-8a44-34d641546b7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c371a7a764c4dc19c8d481f6119684c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:42.660622 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6e019ed3-d0cf-439a-8a44-34d641546b7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c371a7a764c4dc19c8d481f6119684c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:42.666836 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1356/2697] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:29:42 2022] DELETE /identity/v3/users/1c371a7a764c4dc19c8d481f6119684c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:42.671602 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b9642-aaf4-47c6-9be9-9896291b0ad6 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:42.671922 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b9642-aaf4-47c6-9be9-9896291b0ad6 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:42.672193 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b9642-aaf4-47c6-9be9-9896291b0ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:42.692430 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-fa8b9642-aaf4-47c6-9be9-9896291b0ad6 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:29:42.693644 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-fa8b9642-aaf4-47c6-9be9-9896291b0ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:42.755826 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1342/2698] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:29:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:29:42.766466 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e019ed3-d0cf-439a-8a44-34d641546b7f admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:42.768622 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6e019ed3-d0cf-439a-8a44-34d641546b7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:42.777213 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:42.842186 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e019ed3-d0cf-439a-8a44-34d641546b7f admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:42.847307 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1d6e6-2b52-4f8f-a997-ef3af75a5a90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:42.847922 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1d6e6-2b52-4f8f-a997-ef3af75a5a90 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:42.848197 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1d6e6-2b52-4f8f-a997-ef3af75a5a90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:42.848451 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1d6e6-2b52-4f8f-a997-ef3af75a5a90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:42.858801 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b1d6e6-2b52-4f8f-a997-ef3af75a5a90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:42.860222 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b1d6e6-2b52-4f8f-a997-ef3af75a5a90 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:42.869900 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1357/2699] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:29:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:29:43.154156 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8b9642-aaf4-47c6-9be9-9896291b0ad6 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:43.155737 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-fa8b9642-aaf4-47c6-9be9-9896291b0ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:43.168989 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8b9642-aaf4-47c6-9be9-9896291b0ad6 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:43.174726 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a391a243-3a59-48db-9fd5-c2ff5bafd668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:43.175502 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a391a243-3a59-48db-9fd5-c2ff5bafd668 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:43.175931 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a391a243-3a59-48db-9fd5-c2ff5bafd668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:43.176267 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a391a243-3a59-48db-9fd5-c2ff5bafd668 admin admin] PATH_INFO: `/v3/projects/2ab062e3d79a48bc8cf071568fb60329` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:43.177619 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a391a243-3a59-48db-9fd5-c2ff5bafd668 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ab062e3d79a48bc8cf071568fb60329)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:43.179607 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a391a243-3a59-48db-9fd5-c2ff5bafd668 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:43.233207 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a391a243-3a59-48db-9fd5-c2ff5bafd668 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ab062e3d79a48bc8cf071568fb60329', 'request_id': 'req-a391a243-3a59-48db-9fd5-c2ff5bafd668'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:43.239840 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a391a243-3a59-48db-9fd5-c2ff5bafd668 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ab062e3d79a48bc8cf071568fb60329', 'request_id': 'req-a391a243-3a59-48db-9fd5-c2ff5bafd668'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:43.246030 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a391a243-3a59-48db-9fd5-c2ff5bafd668 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ab062e3d79a48bc8cf071568fb60329', 'request_id': 'req-a391a243-3a59-48db-9fd5-c2ff5bafd668'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:43.259862 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a391a243-3a59-48db-9fd5-c2ff5bafd668 admin admin] The token cache is being invalidate because project 2ab062e3d79a48bc8cf071568fb60329 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:29:43.260147 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a391a243-3a59-48db-9fd5-c2ff5bafd668 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:43.260758 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-a391a243-3a59-48db-9fd5-c2ff5bafd668 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:43.262377 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1343/2700] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:29:43 2022] DELETE /identity/v3/projects/2ab062e3d79a48bc8cf071568fb60329 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:43.266672 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1d6e6-2b52-4f8f-a997-ef3af75a5a90 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:43.276642 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1d6e6-2b52-4f8f-a997-ef3af75a5a90 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:43.332309 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3233ef8-ed26-41db-a35a-080c36523c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:43.332924 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3233ef8-ed26-41db-a35a-080c36523c85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:43.333197 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3233ef8-ed26-41db-a35a-080c36523c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:43.333462 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3233ef8-ed26-41db-a35a-080c36523c85 admin admin] PATH_INFO: `/v3/users/cb59e6aecf214cd5a23202bc940097cc` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:43.334627 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3233ef8-ed26-41db-a35a-080c36523c85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb59e6aecf214cd5a23202bc940097cc)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:43.336533 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3233ef8-ed26-41db-a35a-080c36523c85 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:43.385241 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c3233ef8-ed26-41db-a35a-080c36523c85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb59e6aecf214cd5a23202bc940097cc'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:43.393187 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c3233ef8-ed26-41db-a35a-080c36523c85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb59e6aecf214cd5a23202bc940097cc'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:43.401296 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c3233ef8-ed26-41db-a35a-080c36523c85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb59e6aecf214cd5a23202bc940097cc'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:43.401916 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c3233ef8-ed26-41db-a35a-080c36523c85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb59e6aecf214cd5a23202bc940097cc'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:43.414423 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c3233ef8-ed26-41db-a35a-080c36523c85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb59e6aecf214cd5a23202bc940097cc'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:43.420219 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1358/2701] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:29:43 2022] DELETE /identity/v3/users/cb59e6aecf214cd5a23202bc940097cc => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:43.706550 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a391a243-3a59-48db-9fd5-c2ff5bafd668 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:43.708104 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a391a243-3a59-48db-9fd5-c2ff5bafd668 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:43.723922 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a391a243-3a59-48db-9fd5-c2ff5bafd668 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:43.776141 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:43.787482 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75df6078-a68f-4b01-85a7-a53e3bf73bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:43.788148 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df6078-a68f-4b01-85a7-a53e3bf73bb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:43.788433 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df6078-a68f-4b01-85a7-a53e3bf73bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:43.788714 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df6078-a68f-4b01-85a7-a53e3bf73bb1 admin admin] PATH_INFO: `/v3/projects/23a14c9aa55949959f6c07883ed02ba8` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:43.790029 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75df6078-a68f-4b01-85a7-a53e3bf73bb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23a14c9aa55949959f6c07883ed02ba8)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:43.792039 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75df6078-a68f-4b01-85a7-a53e3bf73bb1 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:43.839630 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-75df6078-a68f-4b01-85a7-a53e3bf73bb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23a14c9aa55949959f6c07883ed02ba8', 'request_id': 'req-75df6078-a68f-4b01-85a7-a53e3bf73bb1'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:43.847037 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-75df6078-a68f-4b01-85a7-a53e3bf73bb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23a14c9aa55949959f6c07883ed02ba8', 'request_id': 'req-75df6078-a68f-4b01-85a7-a53e3bf73bb1'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:43.853541 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-75df6078-a68f-4b01-85a7-a53e3bf73bb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23a14c9aa55949959f6c07883ed02ba8', 'request_id': 'req-75df6078-a68f-4b01-85a7-a53e3bf73bb1'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:43.868824 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-75df6078-a68f-4b01-85a7-a53e3bf73bb1 admin admin] The token cache is being invalidate because project 23a14c9aa55949959f6c07883ed02ba8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:29:43.869110 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-75df6078-a68f-4b01-85a7-a53e3bf73bb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:43.869733 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-75df6078-a68f-4b01-85a7-a53e3bf73bb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:43.871278 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1344/2702] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:29:43 2022] DELETE /identity/v3/projects/23a14c9aa55949959f6c07883ed02ba8 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:43.877682 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfb2db-420e-41c5-beec-9751d3ac7249 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:43.878223 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfb2db-420e-41c5-beec-9751d3ac7249 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:43.878223 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfb2db-420e-41c5-beec-9751d3ac7249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:43.899235 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-0ddfb2db-420e-41c5-beec-9751d3ac7249 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:29:43.900457 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0ddfb2db-420e-41c5-beec-9751d3ac7249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:43.920411 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:43.963497 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1359/2703] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:29:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:29:43.973061 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a378cc4-cca0-4094-b2fc-d7278e4d456f None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:43.973345 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a378cc4-cca0-4094-b2fc-d7278e4d456f None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:43.973610 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a378cc4-cca0-4094-b2fc-d7278e4d456f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:43.994476 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-3a378cc4-cca0-4094-b2fc-d7278e4d456f None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:29:43.995695 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3a378cc4-cca0-4094-b2fc-d7278e4d456f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:44.005144 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1345/2704] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:29:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:29:44.009630 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddfb2db-420e-41c5-beec-9751d3ac7249 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:44.020144 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddfb2db-420e-41c5-beec-9751d3ac7249 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:44.025331 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d8d3d6-3ad6-4ee0-b381-b34e787ff9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:44.025935 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8d3d6-3ad6-4ee0-b381-b34e787ff9f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:44.026261 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8d3d6-3ad6-4ee0-b381-b34e787ff9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:44.026543 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8d3d6-3ad6-4ee0-b381-b34e787ff9f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:44.027187 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d8d3d6-3ad6-4ee0-b381-b34e787ff9f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:29:44.027489 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d8d3d6-3ad6-4ee0-b381-b34e787ff9f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:44.029092 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d8d3d6-3ad6-4ee0-b381-b34e787ff9f6 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:44.040963 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1360/2705] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:29:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:29:44.045368 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a378cc4-cca0-4094-b2fc-d7278e4d456f None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:44.056581 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a378cc4-cca0-4094-b2fc-d7278e4d456f None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:44.064197 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f2b44-5780-42bd-aa30-e18eb916bc75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:44.065010 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f2b44-5780-42bd-aa30-e18eb916bc75 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:44.065350 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f2b44-5780-42bd-aa30-e18eb916bc75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:44.065786 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f2b44-5780-42bd-aa30-e18eb916bc75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:44.066724 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98f2b44-5780-42bd-aa30-e18eb916bc75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:44.069636 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98f2b44-5780-42bd-aa30-e18eb916bc75 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:44.102667 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1346/2706] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:29:44 2022] POST /identity/v3/projects => generated 359 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:29:44.106993 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d8d3d6-3ad6-4ee0-b381-b34e787ff9f6 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:44.117147 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d8d3d6-3ad6-4ee0-b381-b34e787ff9f6 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:44.172458 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c81c1e6-8fb0-46c9-b8a3-5c533c88a9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:44.173055 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c81c1e6-8fb0-46c9-b8a3-5c533c88a9e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:44.173331 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c81c1e6-8fb0-46c9-b8a3-5c533c88a9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:44.173597 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c81c1e6-8fb0-46c9-b8a3-5c533c88a9e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:44.174279 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c81c1e6-8fb0-46c9-b8a3-5c533c88a9e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:44.176177 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c81c1e6-8fb0-46c9-b8a3-5c533c88a9e2 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:44.199310 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1361/2707] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:29:44 2022] POST /identity/v3/users => generated 280 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:29:44.203650 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f2b44-5780-42bd-aa30-e18eb916bc75 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:44.213844 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f2b44-5780-42bd-aa30-e18eb916bc75 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:44.218972 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f3531-5db5-424e-89dd-f5e12dd1d6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:44.219909 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f3531-5db5-424e-89dd-f5e12dd1d6a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:44.220176 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f3531-5db5-424e-89dd-f5e12dd1d6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:44.220422 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f3531-5db5-424e-89dd-f5e12dd1d6a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:44.221061 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3f3531-5db5-424e-89dd-f5e12dd1d6a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:44.222743 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3f3531-5db5-424e-89dd-f5e12dd1d6a5 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:44.233038 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1347/2708] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:29:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:29:44.236939 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c81c1e6-8fb0-46c9-b8a3-5c533c88a9e2 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:44.247608 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c81c1e6-8fb0-46c9-b8a3-5c533c88a9e2 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:44.253273 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e5b09-8e9a-4edf-b315-b0e0e6a484fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:44.253870 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e5b09-8e9a-4edf-b315-b0e0e6a484fe admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:44.254196 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e5b09-8e9a-4edf-b315-b0e0e6a484fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:44.254462 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e5b09-8e9a-4edf-b315-b0e0e6a484fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:44.255102 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0e5b09-8e9a-4edf-b315-b0e0e6a484fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:44.256701 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0e5b09-8e9a-4edf-b315-b0e0e6a484fe admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:44.267698 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1362/2709] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:29:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:29:44.272058 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f3531-5db5-424e-89dd-f5e12dd1d6a5 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:44.282087 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f3531-5db5-424e-89dd-f5e12dd1d6a5 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:44.286879 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6329454-db91-49f0-8a8a-7990910115c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:44.287533 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6329454-db91-49f0-8a8a-7990910115c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:44.287803 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6329454-db91-49f0-8a8a-7990910115c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:44.288067 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6329454-db91-49f0-8a8a-7990910115c3 admin admin] PATH_INFO: `/v3/projects/065b5c576c5f401bada283c6c09e6eb7/users/8a7df16835234692b00f4d4979dfd765/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:44.302930 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6329454-db91-49f0-8a8a-7990910115c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=065b5c576c5f401bada283c6c09e6eb7, user_id=8a7df16835234692b00f4d4979dfd765, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:44.305620 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6329454-db91-49f0-8a8a-7990910115c3 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:44.319350 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1348/2710] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:29:44 2022] PUT /identity/v3/projects/065b5c576c5f401bada283c6c09e6eb7/users/8a7df16835234692b00f4d4979dfd765/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:29:44.331380 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606dd508-a71a-4964-888c-13ea58d115d3 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:44.331942 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606dd508-a71a-4964-888c-13ea58d115d3 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:44.332440 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606dd508-a71a-4964-888c-13ea58d115d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:44.365661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-606dd508-a71a-4964-888c-13ea58d115d3 None None] MFA Rules not processed for user `8a7df16835234692b00f4d4979dfd765`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:29:44.366909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-606dd508-a71a-4964-888c-13ea58d115d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:44.445919 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1363/2711] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:29:44 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:29:44.479328 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6329454-db91-49f0-8a8a-7990910115c3 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:44.481077 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a6329454-db91-49f0-8a8a-7990910115c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:44.537782 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6329454-db91-49f0-8a8a-7990910115c3 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:44.541832 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151168b9-632e-484b-9d3d-d456f2fa0644 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:44.542485 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151168b9-632e-484b-9d3d-d456f2fa0644 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:44.542782 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151168b9-632e-484b-9d3d-d456f2fa0644 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:44.543038 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151168b9-632e-484b-9d3d-d456f2fa0644 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:44.551536 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151168b9-632e-484b-9d3d-d456f2fa0644 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:44.553003 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151168b9-632e-484b-9d3d-d456f2fa0644 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:44.561727 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1349/2712] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:29:44 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:29:44.578072 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606dd508-a71a-4964-888c-13ea58d115d3 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:44.600216 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:44.645734 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606dd508-a71a-4964-888c-13ea58d115d3 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:44.650590 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f464ee-1289-44e4-8471-2a3d1b39a5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:44.651219 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f464ee-1289-44e4-8471-2a3d1b39a5f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:44.651460 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f464ee-1289-44e4-8471-2a3d1b39a5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:44.651752 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f464ee-1289-44e4-8471-2a3d1b39a5f5 admin admin] PATH_INFO: `/v3/users/8a7df16835234692b00f4d4979dfd765` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:44.652902 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f464ee-1289-44e4-8471-2a3d1b39a5f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a7df16835234692b00f4d4979dfd765)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:44.654877 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f464ee-1289-44e4-8471-2a3d1b39a5f5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:44.704356 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-32f464ee-1289-44e4-8471-2a3d1b39a5f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a7df16835234692b00f4d4979dfd765'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:44.711334 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-32f464ee-1289-44e4-8471-2a3d1b39a5f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a7df16835234692b00f4d4979dfd765'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:44.721179 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-32f464ee-1289-44e4-8471-2a3d1b39a5f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a7df16835234692b00f4d4979dfd765'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:44.722128 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-32f464ee-1289-44e4-8471-2a3d1b39a5f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a7df16835234692b00f4d4979dfd765'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:44.737500 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-32f464ee-1289-44e4-8471-2a3d1b39a5f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a7df16835234692b00f4d4979dfd765'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:44.744102 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1364/2713] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:29:44 2022] DELETE /identity/v3/users/8a7df16835234692b00f4d4979dfd765 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:44.748981 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e94bda-cf0a-442f-8ac2-97bb68241e78 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:44.749290 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e94bda-cf0a-442f-8ac2-97bb68241e78 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:44.749557 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e94bda-cf0a-442f-8ac2-97bb68241e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:44.769374 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-c0e94bda-cf0a-442f-8ac2-97bb68241e78 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:29:44.770617 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c0e94bda-cf0a-442f-8ac2-97bb68241e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:44.787939 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:44.835779 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1350/2714] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:29:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:29:44.846261 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f464ee-1289-44e4-8471-2a3d1b39a5f5 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:44.847786 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-32f464ee-1289-44e4-8471-2a3d1b39a5f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:44.907140 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f464ee-1289-44e4-8471-2a3d1b39a5f5 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:44.911846 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607c35f0-3436-4051-92f7-f82e9cf2eff1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:44.912480 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c35f0-3436-4051-92f7-f82e9cf2eff1 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:44.912756 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c35f0-3436-4051-92f7-f82e9cf2eff1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:44.913022 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c35f0-3436-4051-92f7-f82e9cf2eff1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:44.923063 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607c35f0-3436-4051-92f7-f82e9cf2eff1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:44.924495 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607c35f0-3436-4051-92f7-f82e9cf2eff1 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:44.934468 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1365/2715] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:29:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:29:45.217400 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e94bda-cf0a-442f-8ac2-97bb68241e78 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:45.218974 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c0e94bda-cf0a-442f-8ac2-97bb68241e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:45.232166 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e94bda-cf0a-442f-8ac2-97bb68241e78 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:45.238652 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b5609-9ad6-46de-a630-a3c9b0afe2ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:45.239312 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b5609-9ad6-46de-a630-a3c9b0afe2ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:45.239591 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b5609-9ad6-46de-a630-a3c9b0afe2ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:45.239831 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b5609-9ad6-46de-a630-a3c9b0afe2ab admin admin] PATH_INFO: `/v3/projects/065b5c576c5f401bada283c6c09e6eb7` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:45.241085 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066b5609-9ad6-46de-a630-a3c9b0afe2ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=065b5c576c5f401bada283c6c09e6eb7)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:45.243064 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066b5609-9ad6-46de-a630-a3c9b0afe2ab admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:45.301361 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-066b5609-9ad6-46de-a630-a3c9b0afe2ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '065b5c576c5f401bada283c6c09e6eb7', 'request_id': 'req-066b5609-9ad6-46de-a630-a3c9b0afe2ab'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:45.308450 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-066b5609-9ad6-46de-a630-a3c9b0afe2ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '065b5c576c5f401bada283c6c09e6eb7', 'request_id': 'req-066b5609-9ad6-46de-a630-a3c9b0afe2ab'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:45.314950 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-066b5609-9ad6-46de-a630-a3c9b0afe2ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '065b5c576c5f401bada283c6c09e6eb7', 'request_id': 'req-066b5609-9ad6-46de-a630-a3c9b0afe2ab'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:45.329694 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-066b5609-9ad6-46de-a630-a3c9b0afe2ab admin admin] The token cache is being invalidate because project 065b5c576c5f401bada283c6c09e6eb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:29:45.329976 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-066b5609-9ad6-46de-a630-a3c9b0afe2ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:45.330664 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-066b5609-9ad6-46de-a630-a3c9b0afe2ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:45.332195 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1351/2716] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:29:45 2022] DELETE /identity/v3/projects/065b5c576c5f401bada283c6c09e6eb7 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:45.337267 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d466519-5251-49e1-98ec-7ff653373d94 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:45.337553 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d466519-5251-49e1-98ec-7ff653373d94 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:45.337824 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d466519-5251-49e1-98ec-7ff653373d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:45.357550 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-9d466519-5251-49e1-98ec-7ff653373d94 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:29:45.358786 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9d466519-5251-49e1-98ec-7ff653373d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:45.406495 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:45.433551 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1366/2717] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:29:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:29:45.441695 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0816c2-0b12-40ab-a31c-dd0e9a8b2aaf None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:45.441967 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0816c2-0b12-40ab-a31c-dd0e9a8b2aaf None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:45.442329 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0816c2-0b12-40ab-a31c-dd0e9a8b2aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:45.463713 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-9b0816c2-0b12-40ab-a31c-dd0e9a8b2aaf None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:29:45.464907 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-9b0816c2-0b12-40ab-a31c-dd0e9a8b2aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:45.473293 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1352/2718] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:29:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:29:45.477696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d466519-5251-49e1-98ec-7ff653373d94 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:45.487770 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d466519-5251-49e1-98ec-7ff653373d94 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:45.492675 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c6cba8-7aaa-4857-9978-1b7f3054cf0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:45.493276 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6cba8-7aaa-4857-9978-1b7f3054cf0b admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:45.493551 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6cba8-7aaa-4857-9978-1b7f3054cf0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:45.493801 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6cba8-7aaa-4857-9978-1b7f3054cf0b admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:45.494483 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c6cba8-7aaa-4857-9978-1b7f3054cf0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:29:45.494933 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c6cba8-7aaa-4857-9978-1b7f3054cf0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:45.496365 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c6cba8-7aaa-4857-9978-1b7f3054cf0b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:45.508370 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1367/2719] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:29:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:29:45.513486 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0816c2-0b12-40ab-a31c-dd0e9a8b2aaf None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:45.523262 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0816c2-0b12-40ab-a31c-dd0e9a8b2aaf None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:45.528582 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86722fc9-00f6-47f7-ac23-1e9fa89c1830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:45.529220 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86722fc9-00f6-47f7-ac23-1e9fa89c1830 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:45.529491 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86722fc9-00f6-47f7-ac23-1e9fa89c1830 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:45.529787 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86722fc9-00f6-47f7-ac23-1e9fa89c1830 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:45.530614 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86722fc9-00f6-47f7-ac23-1e9fa89c1830 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:45.532523 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86722fc9-00f6-47f7-ac23-1e9fa89c1830 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:45.563008 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1353/2720] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:29:45 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:29:45.567194 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c6cba8-7aaa-4857-9978-1b7f3054cf0b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:45.577355 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c6cba8-7aaa-4857-9978-1b7f3054cf0b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:45.634434 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c24e942-a071-41e5-bb36-096b76fe09a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:45.635026 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24e942-a071-41e5-bb36-096b76fe09a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:45.635295 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24e942-a071-41e5-bb36-096b76fe09a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:45.635558 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24e942-a071-41e5-bb36-096b76fe09a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:45.636187 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c24e942-a071-41e5-bb36-096b76fe09a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:45.638138 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c24e942-a071-41e5-bb36-096b76fe09a9 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:45.664785 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1368/2721] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:29:45 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:29:45.671044 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86722fc9-00f6-47f7-ac23-1e9fa89c1830 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:45.684498 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86722fc9-00f6-47f7-ac23-1e9fa89c1830 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:45.693440 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cb78ee-8792-4589-8683-7f79e6dd61b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:45.694371 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb78ee-8792-4589-8683-7f79e6dd61b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:45.694735 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb78ee-8792-4589-8683-7f79e6dd61b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:45.695123 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb78ee-8792-4589-8683-7f79e6dd61b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:45.696124 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cb78ee-8792-4589-8683-7f79e6dd61b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:45.698736 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cb78ee-8792-4589-8683-7f79e6dd61b1 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:45.710334 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1354/2722] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:29:45.714430 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c24e942-a071-41e5-bb36-096b76fe09a9 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:45.725251 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c24e942-a071-41e5-bb36-096b76fe09a9 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:45.731132 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a83be-4e1e-410f-a9aa-770f21c9faa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:45.731978 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a83be-4e1e-410f-a9aa-770f21c9faa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:45.732479 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a83be-4e1e-410f-a9aa-770f21c9faa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:45.732755 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a83be-4e1e-410f-a9aa-770f21c9faa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:45.733398 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6a83be-4e1e-410f-a9aa-770f21c9faa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:45.735065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6a83be-4e1e-410f-a9aa-770f21c9faa0 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:45.745988 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1369/2723] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:29:45.751424 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cb78ee-8792-4589-8683-7f79e6dd61b1 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:45.763092 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cb78ee-8792-4589-8683-7f79e6dd61b1 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:45.768442 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac38d1d-050a-4729-824f-220af71d74ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:45.769118 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac38d1d-050a-4729-824f-220af71d74ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:45.769398 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac38d1d-050a-4729-824f-220af71d74ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:45.769667 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac38d1d-050a-4729-824f-220af71d74ac admin admin] PATH_INFO: `/v3/projects/0b799e132c064654ab216b6f4a649984/users/9edfbd27abf74b19a1e301c0564f99e1/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:45.784629 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac38d1d-050a-4729-824f-220af71d74ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b799e132c064654ab216b6f4a649984, user_id=9edfbd27abf74b19a1e301c0564f99e1, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:45.787358 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac38d1d-050a-4729-824f-220af71d74ac admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:45.802593 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1355/2724] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:29:45 2022] PUT /identity/v3/projects/0b799e132c064654ab216b6f4a649984/users/9edfbd27abf74b19a1e301c0564f99e1/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:45.872659 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f245ebfb-3094-4b65-aac0-c5995e4c7781 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:45.873049 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f245ebfb-3094-4b65-aac0-c5995e4c7781 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:45.873300 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f245ebfb-3094-4b65-aac0-c5995e4c7781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:45.904973 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f245ebfb-3094-4b65-aac0-c5995e4c7781 None None] MFA Rules not processed for user `9edfbd27abf74b19a1e301c0564f99e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:29:45.906267 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f245ebfb-3094-4b65-aac0-c5995e4c7781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:45.973786 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1370/2725] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:29:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:29:45.978742 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac38d1d-050a-4729-824f-220af71d74ac admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:46.024965 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:46.044578 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac38d1d-050a-4729-824f-220af71d74ac admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:46.049597 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4280a81-62e4-47ac-a33c-48d778e9b3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:46.050223 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4280a81-62e4-47ac-a33c-48d778e9b3ba admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:46.050495 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4280a81-62e4-47ac-a33c-48d778e9b3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:46.050759 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4280a81-62e4-47ac-a33c-48d778e9b3ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:46.051375 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4280a81-62e4-47ac-a33c-48d778e9b3ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:46.053291 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4280a81-62e4-47ac-a33c-48d778e9b3ba admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:46.081355 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1356/2726] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:29:45 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:29:46.085676 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f245ebfb-3094-4b65-aac0-c5995e4c7781 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:46.097390 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f245ebfb-3094-4b65-aac0-c5995e4c7781 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:46.131979 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:46.156814 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f63285f-a3f8-4537-8cf6-08fdfac7b76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:46.157440 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f63285f-a3f8-4537-8cf6-08fdfac7b76e admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:46.157716 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f63285f-a3f8-4537-8cf6-08fdfac7b76e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:46.157976 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f63285f-a3f8-4537-8cf6-08fdfac7b76e admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:46.158653 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f63285f-a3f8-4537-8cf6-08fdfac7b76e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:46.160553 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f63285f-a3f8-4537-8cf6-08fdfac7b76e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:46.186768 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1371/2727] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:29:46 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:29:46.191492 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4280a81-62e4-47ac-a33c-48d778e9b3ba admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:46.203543 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4280a81-62e4-47ac-a33c-48d778e9b3ba admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:46.209819 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ef38f-8ee3-460d-828a-b120412e8031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:46.210443 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ef38f-8ee3-460d-828a-b120412e8031 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:46.210716 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ef38f-8ee3-460d-828a-b120412e8031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:46.211006 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ef38f-8ee3-460d-828a-b120412e8031 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:46.211648 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291ef38f-8ee3-460d-828a-b120412e8031 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:46.213277 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291ef38f-8ee3-460d-828a-b120412e8031 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:46.227582 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1357/2728] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:29:46.231639 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f63285f-a3f8-4537-8cf6-08fdfac7b76e admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:46.241826 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f63285f-a3f8-4537-8cf6-08fdfac7b76e admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:46.246966 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb14ec0-6baf-4d11-8dd0-8ea50a21e9b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:46.247552 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb14ec0-6baf-4d11-8dd0-8ea50a21e9b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:46.247825 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb14ec0-6baf-4d11-8dd0-8ea50a21e9b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:46.248087 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb14ec0-6baf-4d11-8dd0-8ea50a21e9b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:46.248722 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb14ec0-6baf-4d11-8dd0-8ea50a21e9b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:46.250326 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb14ec0-6baf-4d11-8dd0-8ea50a21e9b0 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:46.260431 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1372/2729] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:29:46.264794 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ef38f-8ee3-460d-828a-b120412e8031 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:46.275151 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ef38f-8ee3-460d-828a-b120412e8031 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:46.280110 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7565b5-f6ab-48d2-8038-5af1ce110205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:46.280753 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7565b5-f6ab-48d2-8038-5af1ce110205 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:46.281033 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7565b5-f6ab-48d2-8038-5af1ce110205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:46.281278 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7565b5-f6ab-48d2-8038-5af1ce110205 admin admin] PATH_INFO: `/v3/projects/a90e7d38156e42998ebceea0ecd6d25b/users/69b30a834afb4b849951d5b7e47abd5a/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:46.296517 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7565b5-f6ab-48d2-8038-5af1ce110205 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a90e7d38156e42998ebceea0ecd6d25b, user_id=69b30a834afb4b849951d5b7e47abd5a, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:46.299238 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7565b5-f6ab-48d2-8038-5af1ce110205 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:46.314110 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1358/2730] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:29:46 2022] PUT /identity/v3/projects/a90e7d38156e42998ebceea0ecd6d25b/users/69b30a834afb4b849951d5b7e47abd5a/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:46.324542 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92abfd68-eaaa-481c-b299-36898969ca76 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:46.324828 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92abfd68-eaaa-481c-b299-36898969ca76 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:46.325095 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92abfd68-eaaa-481c-b299-36898969ca76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:46.354650 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-92abfd68-eaaa-481c-b299-36898969ca76 None None] MFA Rules not processed for user `69b30a834afb4b849951d5b7e47abd5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:29:46.355896 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-92abfd68-eaaa-481c-b299-36898969ca76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:46.423202 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1373/2731] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:29:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:29:46.430295 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7565b5-f6ab-48d2-8038-5af1ce110205 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:46.501259 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7565b5-f6ab-48d2-8038-5af1ce110205 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:46.506209 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc5a98-0152-4303-8da5-8368b00197f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:46.506825 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc5a98-0152-4303-8da5-8368b00197f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:46.507097 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc5a98-0152-4303-8da5-8368b00197f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:46.507381 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc5a98-0152-4303-8da5-8368b00197f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:46.508008 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bc5a98-0152-4303-8da5-8368b00197f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:46.509925 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bc5a98-0152-4303-8da5-8368b00197f7 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:46.538379 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1359/2732] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:29:46 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:29:46.543247 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92abfd68-eaaa-481c-b299-36898969ca76 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:46.556302 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92abfd68-eaaa-481c-b299-36898969ca76 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:46.614758 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f9a888-1e03-49e1-b388-121b8e83b77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:46.615398 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f9a888-1e03-49e1-b388-121b8e83b77b admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:46.615670 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f9a888-1e03-49e1-b388-121b8e83b77b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:46.615935 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f9a888-1e03-49e1-b388-121b8e83b77b admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:46.616556 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f9a888-1e03-49e1-b388-121b8e83b77b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:46.618499 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f9a888-1e03-49e1-b388-121b8e83b77b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:46.642705 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1374/2733] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:29:46 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:29:46.646794 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc5a98-0152-4303-8da5-8368b00197f7 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:46.657725 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc5a98-0152-4303-8da5-8368b00197f7 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:46.662611 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf035e62-17f0-4787-9a53-af6eb9491fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:46.663168 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf035e62-17f0-4787-9a53-af6eb9491fd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:46.663448 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf035e62-17f0-4787-9a53-af6eb9491fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:46.663721 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf035e62-17f0-4787-9a53-af6eb9491fd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:46.664371 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf035e62-17f0-4787-9a53-af6eb9491fd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:46.666054 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf035e62-17f0-4787-9a53-af6eb9491fd2 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:46.677108 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1360/2734] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:29:46.681464 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f9a888-1e03-49e1-b388-121b8e83b77b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:46.691707 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f9a888-1e03-49e1-b388-121b8e83b77b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:46.696716 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b8ac7-6d36-4e05-af1f-a9e158fcbbf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:46.697358 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b8ac7-6d36-4e05-af1f-a9e158fcbbf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:46.697628 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b8ac7-6d36-4e05-af1f-a9e158fcbbf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:46.697889 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b8ac7-6d36-4e05-af1f-a9e158fcbbf7 admin admin] PATH_INFO: `/v3/projects/13abac61629e430ebd2e5be224dbdee6/users/26211b0cdfb541ceaa628d32a5a95e9c/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:46.712881 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8b8ac7-6d36-4e05-af1f-a9e158fcbbf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13abac61629e430ebd2e5be224dbdee6, user_id=26211b0cdfb541ceaa628d32a5a95e9c, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:46.715544 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8b8ac7-6d36-4e05-af1f-a9e158fcbbf7 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:46.736305 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1375/2735] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:29:46 2022] PUT /identity/v3/projects/13abac61629e430ebd2e5be224dbdee6/users/26211b0cdfb541ceaa628d32a5a95e9c/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:29:46.741809 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf035e62-17f0-4787-9a53-af6eb9491fd2 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:46.753343 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf035e62-17f0-4787-9a53-af6eb9491fd2 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:46.813385 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577b0488-b9e6-49c3-9f5e-3db2aaba4469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:46.813993 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b0488-b9e6-49c3-9f5e-3db2aaba4469 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:46.814266 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b0488-b9e6-49c3-9f5e-3db2aaba4469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:46.814614 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b0488-b9e6-49c3-9f5e-3db2aaba4469 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:46.815261 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577b0488-b9e6-49c3-9f5e-3db2aaba4469 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:46.816895 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577b0488-b9e6-49c3-9f5e-3db2aaba4469 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:46.827021 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1361/2736] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:29:46.832015 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b8ac7-6d36-4e05-af1f-a9e158fcbbf7 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:46.844333 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b8ac7-6d36-4e05-af1f-a9e158fcbbf7 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:46.849807 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f866661-9f73-4a99-8228-6065a4c61d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:46.850518 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f866661-9f73-4a99-8228-6065a4c61d78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:46.850797 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f866661-9f73-4a99-8228-6065a4c61d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:46.851064 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f866661-9f73-4a99-8228-6065a4c61d78 admin admin] PATH_INFO: `/v3/projects/13abac61629e430ebd2e5be224dbdee6/users/26211b0cdfb541ceaa628d32a5a95e9c/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:46.853041 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f866661-9f73-4a99-8228-6065a4c61d78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13abac61629e430ebd2e5be224dbdee6, user_id=26211b0cdfb541ceaa628d32a5a95e9c, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:46.855845 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f866661-9f73-4a99-8228-6065a4c61d78 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:46.861490 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:46.867236 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1376/2737] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:29:46 2022] PUT /identity/v3/projects/13abac61629e430ebd2e5be224dbdee6/users/26211b0cdfb541ceaa628d32a5a95e9c/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:46.877862 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed44a3-b79b-4974-be34-ff5191262ce4 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:46.878181 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed44a3-b79b-4974-be34-ff5191262ce4 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:46.878459 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed44a3-b79b-4974-be34-ff5191262ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:46.885762 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:46.909065 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-62ed44a3-b79b-4974-be34-ff5191262ce4 None None] MFA Rules not processed for user `26211b0cdfb541ceaa628d32a5a95e9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:29:46.910344 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-62ed44a3-b79b-4974-be34-ff5191262ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:46.982714 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1362/2738] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:29:46 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:29:46.995918 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f866661-9f73-4a99-8228-6065a4c61d78 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:46.997872 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2f866661-9f73-4a99-8228-6065a4c61d78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:47.060974 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f866661-9f73-4a99-8228-6065a4c61d78 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:47.066124 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b307ed-3374-4398-9971-74c418bf7652 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:47.066723 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b307ed-3374-4398-9971-74c418bf7652 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:47.066999 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b307ed-3374-4398-9971-74c418bf7652 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:47.067262 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b307ed-3374-4398-9971-74c418bf7652 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:47.124662 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b307ed-3374-4398-9971-74c418bf7652 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:47.126134 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b307ed-3374-4398-9971-74c418bf7652 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:47.135035 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1377/2739] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:29:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:29:49.716519 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed44a3-b79b-4974-be34-ff5191262ce4 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:49.727652 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed44a3-b79b-4974-be34-ff5191262ce4 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:49.731866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd114993-ddfc-4a22-aaa1-19c5f198d98e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:49.732479 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd114993-ddfc-4a22-aaa1-19c5f198d98e service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:49.732756 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd114993-ddfc-4a22-aaa1-19c5f198d98e service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:49.733023 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd114993-ddfc-4a22-aaa1-19c5f198d98e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:49.790340 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd114993-ddfc-4a22-aaa1-19c5f198d98e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:49.791793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd114993-ddfc-4a22-aaa1-19c5f198d98e service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:49.800588 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1363/2740] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:29:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:29:50.354471 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b307ed-3374-4398-9971-74c418bf7652 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:50.363727 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b307ed-3374-4398-9971-74c418bf7652 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:50.368135 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda71d0c-077f-460c-b3b1-87171f37d605 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:50.368741 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda71d0c-077f-460c-b3b1-87171f37d605 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:50.369020 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda71d0c-077f-460c-b3b1-87171f37d605 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:50.369287 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda71d0c-077f-460c-b3b1-87171f37d605 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:50.379376 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda71d0c-077f-460c-b3b1-87171f37d605 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:50.380796 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda71d0c-077f-460c-b3b1-87171f37d605 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:50.390151 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1378/2741] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:29:50 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:29:55.384893 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd114993-ddfc-4a22-aaa1-19c5f198d98e service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:55.450639 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd114993-ddfc-4a22-aaa1-19c5f198d98e service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:55.455190 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0600a5-1aa4-427c-adad-f96815171e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:55.455839 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0600a5-1aa4-427c-adad-f96815171e7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:55.456113 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0600a5-1aa4-427c-adad-f96815171e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:55.456378 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0600a5-1aa4-427c-adad-f96815171e7d admin admin] PATH_INFO: `/v3/users/9edfbd27abf74b19a1e301c0564f99e1` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:55.457515 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0600a5-1aa4-427c-adad-f96815171e7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9edfbd27abf74b19a1e301c0564f99e1)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:55.459509 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0600a5-1aa4-427c-adad-f96815171e7d admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:55.489994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:55.509213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-4f0600a5-1aa4-427c-adad-f96815171e7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9edfbd27abf74b19a1e301c0564f99e1'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:55.516142 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-4f0600a5-1aa4-427c-adad-f96815171e7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9edfbd27abf74b19a1e301c0564f99e1'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:55.516826 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-4f0600a5-1aa4-427c-adad-f96815171e7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9edfbd27abf74b19a1e301c0564f99e1'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:55.525988 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-4f0600a5-1aa4-427c-adad-f96815171e7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9edfbd27abf74b19a1e301c0564f99e1'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:55.526713 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-4f0600a5-1aa4-427c-adad-f96815171e7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9edfbd27abf74b19a1e301c0564f99e1'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:55.544634 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1364/2742] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:29:55 2022] DELETE /identity/v3/users/9edfbd27abf74b19a1e301c0564f99e1 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:55.549117 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1729ed-44ed-4a47-89ba-57bc69f6eeee None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:55.549643 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1729ed-44ed-4a47-89ba-57bc69f6eeee None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:55.549643 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1729ed-44ed-4a47-89ba-57bc69f6eeee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:55.569160 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-3f1729ed-44ed-4a47-89ba-57bc69f6eeee None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:29:55.570410 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3f1729ed-44ed-4a47-89ba-57bc69f6eeee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:55.629576 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1379/2743] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:29:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:29:55.640099 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0600a5-1aa4-427c-adad-f96815171e7d admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:55.641749 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-4f0600a5-1aa4-427c-adad-f96815171e7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:55.708666 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0600a5-1aa4-427c-adad-f96815171e7d admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:55.712653 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc9cec-026f-4b00-b670-e04a0ec4acf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:55.713262 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc9cec-026f-4b00-b670-e04a0ec4acf8 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:55.713533 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc9cec-026f-4b00-b670-e04a0ec4acf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:55.713794 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc9cec-026f-4b00-b670-e04a0ec4acf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:55.725222 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bc9cec-026f-4b00-b670-e04a0ec4acf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:55.725222 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bc9cec-026f-4b00-b670-e04a0ec4acf8 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:55.737487 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1365/2744] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:29:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:29:56.011318 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1729ed-44ed-4a47-89ba-57bc69f6eeee None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:56.012916 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3f1729ed-44ed-4a47-89ba-57bc69f6eeee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:56.026619 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1729ed-44ed-4a47-89ba-57bc69f6eeee None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:56.031945 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a9f98f-2aa2-4c0d-8a5d-905fb86e707c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:56.032671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a9f98f-2aa2-4c0d-8a5d-905fb86e707c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:56.033026 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a9f98f-2aa2-4c0d-8a5d-905fb86e707c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:56.033358 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a9f98f-2aa2-4c0d-8a5d-905fb86e707c admin admin] PATH_INFO: `/v3/projects/0b799e132c064654ab216b6f4a649984` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:56.035341 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a9f98f-2aa2-4c0d-8a5d-905fb86e707c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b799e132c064654ab216b6f4a649984)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:56.037497 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a9f98f-2aa2-4c0d-8a5d-905fb86e707c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:56.076866 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:56.087024 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c0a9f98f-2aa2-4c0d-8a5d-905fb86e707c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b799e132c064654ab216b6f4a649984', 'request_id': 'req-c0a9f98f-2aa2-4c0d-8a5d-905fb86e707c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:56.093080 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c0a9f98f-2aa2-4c0d-8a5d-905fb86e707c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b799e132c064654ab216b6f4a649984', 'request_id': 'req-c0a9f98f-2aa2-4c0d-8a5d-905fb86e707c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:56.101649 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c0a9f98f-2aa2-4c0d-8a5d-905fb86e707c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b799e132c064654ab216b6f4a649984', 'request_id': 'req-c0a9f98f-2aa2-4c0d-8a5d-905fb86e707c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:56.112294 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c0a9f98f-2aa2-4c0d-8a5d-905fb86e707c admin admin] The token cache is being invalidate because project 0b799e132c064654ab216b6f4a649984 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:29:56.112576 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c0a9f98f-2aa2-4c0d-8a5d-905fb86e707c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:56.113220 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c0a9f98f-2aa2-4c0d-8a5d-905fb86e707c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:56.114823 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1380/2745] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:29:56 2022] DELETE /identity/v3/projects/0b799e132c064654ab216b6f4a649984 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:56.119143 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc9cec-026f-4b00-b670-e04a0ec4acf8 service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:56.129235 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc9cec-026f-4b00-b670-e04a0ec4acf8 service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:56.189954 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28878313-3dd7-4d71-a9e2-d6e0ff54a8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:56.190617 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28878313-3dd7-4d71-a9e2-d6e0ff54a8ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:56.190895 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28878313-3dd7-4d71-a9e2-d6e0ff54a8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:56.191157 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28878313-3dd7-4d71-a9e2-d6e0ff54a8ed admin admin] PATH_INFO: `/v3/users/69b30a834afb4b849951d5b7e47abd5a` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:56.192391 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28878313-3dd7-4d71-a9e2-d6e0ff54a8ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=69b30a834afb4b849951d5b7e47abd5a)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:56.194386 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28878313-3dd7-4d71-a9e2-d6e0ff54a8ed admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:56.241778 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-28878313-3dd7-4d71-a9e2-d6e0ff54a8ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69b30a834afb4b849951d5b7e47abd5a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:56.243762 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:56.249819 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-28878313-3dd7-4d71-a9e2-d6e0ff54a8ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69b30a834afb4b849951d5b7e47abd5a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:56.250579 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-28878313-3dd7-4d71-a9e2-d6e0ff54a8ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69b30a834afb4b849951d5b7e47abd5a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:56.258918 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-28878313-3dd7-4d71-a9e2-d6e0ff54a8ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69b30a834afb4b849951d5b7e47abd5a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:56.259598 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-28878313-3dd7-4d71-a9e2-d6e0ff54a8ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69b30a834afb4b849951d5b7e47abd5a'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:56.276554 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1366/2746] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:29:56 2022] DELETE /identity/v3/users/69b30a834afb4b849951d5b7e47abd5a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:56.554629 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a9f98f-2aa2-4c0d-8a5d-905fb86e707c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:56.556205 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c0a9f98f-2aa2-4c0d-8a5d-905fb86e707c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:56.570316 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a9f98f-2aa2-4c0d-8a5d-905fb86e707c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:56.627913 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c733d7e-c428-424f-a02b-fdba4816b6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:56.628464 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c733d7e-c428-424f-a02b-fdba4816b6f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:56.628740 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c733d7e-c428-424f-a02b-fdba4816b6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:56.629001 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c733d7e-c428-424f-a02b-fdba4816b6f9 admin admin] PATH_INFO: `/v3/projects/a90e7d38156e42998ebceea0ecd6d25b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:56.630243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c733d7e-c428-424f-a02b-fdba4816b6f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a90e7d38156e42998ebceea0ecd6d25b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:56.632127 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c733d7e-c428-424f-a02b-fdba4816b6f9 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:56.682979 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1c733d7e-c428-424f-a02b-fdba4816b6f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a90e7d38156e42998ebceea0ecd6d25b', 'request_id': 'req-1c733d7e-c428-424f-a02b-fdba4816b6f9'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:56.689327 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1c733d7e-c428-424f-a02b-fdba4816b6f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a90e7d38156e42998ebceea0ecd6d25b', 'request_id': 'req-1c733d7e-c428-424f-a02b-fdba4816b6f9'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:56.699459 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1c733d7e-c428-424f-a02b-fdba4816b6f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a90e7d38156e42998ebceea0ecd6d25b', 'request_id': 'req-1c733d7e-c428-424f-a02b-fdba4816b6f9'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:56.712355 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1c733d7e-c428-424f-a02b-fdba4816b6f9 admin admin] The token cache is being invalidate because project a90e7d38156e42998ebceea0ecd6d25b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:29:56.712633 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1c733d7e-c428-424f-a02b-fdba4816b6f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:56.713281 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1c733d7e-c428-424f-a02b-fdba4816b6f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:56.714823 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1381/2747] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:29:56 2022] DELETE /identity/v3/projects/a90e7d38156e42998ebceea0ecd6d25b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:56.719105 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28878313-3dd7-4d71-a9e2-d6e0ff54a8ed admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:56.721022 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-28878313-3dd7-4d71-a9e2-d6e0ff54a8ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:56.734756 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28878313-3dd7-4d71-a9e2-d6e0ff54a8ed admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:56.792683 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ac374-8db0-48f8-86d4-68620ae67090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:56.793308 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ac374-8db0-48f8-86d4-68620ae67090 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:56.793584 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ac374-8db0-48f8-86d4-68620ae67090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:56.793850 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ac374-8db0-48f8-86d4-68620ae67090 admin admin] PATH_INFO: `/v3/users/26211b0cdfb541ceaa628d32a5a95e9c` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:56.795007 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ac374-8db0-48f8-86d4-68620ae67090 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26211b0cdfb541ceaa628d32a5a95e9c)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:56.796937 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ac374-8db0-48f8-86d4-68620ae67090 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:56.849334 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c87ac374-8db0-48f8-86d4-68620ae67090 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26211b0cdfb541ceaa628d32a5a95e9c'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:56.856276 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c87ac374-8db0-48f8-86d4-68620ae67090 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26211b0cdfb541ceaa628d32a5a95e9c'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:56.857079 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c87ac374-8db0-48f8-86d4-68620ae67090 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26211b0cdfb541ceaa628d32a5a95e9c'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:56.865110 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c87ac374-8db0-48f8-86d4-68620ae67090 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26211b0cdfb541ceaa628d32a5a95e9c'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:56.865740 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c87ac374-8db0-48f8-86d4-68620ae67090 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26211b0cdfb541ceaa628d32a5a95e9c'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:56.884124 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1367/2748] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:29:56 2022] DELETE /identity/v3/users/26211b0cdfb541ceaa628d32a5a95e9c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:57.175393 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c733d7e-c428-424f-a02b-fdba4816b6f9 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:57.176956 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1c733d7e-c428-424f-a02b-fdba4816b6f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:57.190164 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c733d7e-c428-424f-a02b-fdba4816b6f9 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:57.240152 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:57.250167 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e261d14-579a-46d3-b30e-920e7ec1c75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:57.250819 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e261d14-579a-46d3-b30e-920e7ec1c75a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:57.251156 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e261d14-579a-46d3-b30e-920e7ec1c75a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:57.251414 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e261d14-579a-46d3-b30e-920e7ec1c75a admin admin] PATH_INFO: `/v3/projects/13abac61629e430ebd2e5be224dbdee6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:57.252662 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e261d14-579a-46d3-b30e-920e7ec1c75a admin admin] RBAC: Authorizing `identity:delete_project(project_id=13abac61629e430ebd2e5be224dbdee6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:57.255032 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e261d14-579a-46d3-b30e-920e7ec1c75a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:57.309393 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7e261d14-579a-46d3-b30e-920e7ec1c75a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13abac61629e430ebd2e5be224dbdee6', 'request_id': 'req-7e261d14-579a-46d3-b30e-920e7ec1c75a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:57.315513 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7e261d14-579a-46d3-b30e-920e7ec1c75a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13abac61629e430ebd2e5be224dbdee6', 'request_id': 'req-7e261d14-579a-46d3-b30e-920e7ec1c75a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:57.326551 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7e261d14-579a-46d3-b30e-920e7ec1c75a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13abac61629e430ebd2e5be224dbdee6', 'request_id': 'req-7e261d14-579a-46d3-b30e-920e7ec1c75a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:57.337223 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7e261d14-579a-46d3-b30e-920e7ec1c75a admin admin] The token cache is being invalidate because project 13abac61629e430ebd2e5be224dbdee6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:29:57.337505 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7e261d14-579a-46d3-b30e-920e7ec1c75a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:57.338176 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7e261d14-579a-46d3-b30e-920e7ec1c75a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:29:57.339774 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1382/2749] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:29:57 2022] DELETE /identity/v3/projects/13abac61629e430ebd2e5be224dbdee6 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:29:57.345255 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26094272-02f2-47fa-8e12-9921c22f3e02 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:57.345542 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26094272-02f2-47fa-8e12-9921c22f3e02 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:57.345818 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26094272-02f2-47fa-8e12-9921c22f3e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:57.366989 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-26094272-02f2-47fa-8e12-9921c22f3e02 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:29:57.368204 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-26094272-02f2-47fa-8e12-9921c22f3e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:57.411635 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:29:57.432668 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1368/2750] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:29:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:29:57.440561 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4814095-b9cf-4991-8d19-cdf2f69b7d4b None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:57.440843 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4814095-b9cf-4991-8d19-cdf2f69b7d4b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:57.441112 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4814095-b9cf-4991-8d19-cdf2f69b7d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:57.460929 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-c4814095-b9cf-4991-8d19-cdf2f69b7d4b None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:29:57.462139 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c4814095-b9cf-4991-8d19-cdf2f69b7d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:57.471131 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1383/2751] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:29:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:29:57.475675 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26094272-02f2-47fa-8e12-9921c22f3e02 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:57.486475 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26094272-02f2-47fa-8e12-9921c22f3e02 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:57.492630 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac4a55f-815b-4437-a2ea-4326a2773578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:57.493235 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4a55f-815b-4437-a2ea-4326a2773578 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:57.493509 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4a55f-815b-4437-a2ea-4326a2773578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:57.493773 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4a55f-815b-4437-a2ea-4326a2773578 admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:57.494462 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac4a55f-815b-4437-a2ea-4326a2773578 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:29:57.494760 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac4a55f-815b-4437-a2ea-4326a2773578 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:57.496382 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac4a55f-815b-4437-a2ea-4326a2773578 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:57.510908 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1369/2752] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:29:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:29:57.513584 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4814095-b9cf-4991-8d19-cdf2f69b7d4b None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:57.523437 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4814095-b9cf-4991-8d19-cdf2f69b7d4b None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:57.528715 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa454c-c988-4d97-9b84-df7903eb02f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:57.529332 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa454c-c988-4d97-9b84-df7903eb02f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:57.529606 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa454c-c988-4d97-9b84-df7903eb02f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:57.529870 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa454c-c988-4d97-9b84-df7903eb02f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:57.530528 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aa454c-c988-4d97-9b84-df7903eb02f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:57.532419 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aa454c-c988-4d97-9b84-df7903eb02f4 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:57.561837 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1384/2753] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:29:57 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:29:57.566101 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac4a55f-815b-4437-a2ea-4326a2773578 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:57.575974 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac4a55f-815b-4437-a2ea-4326a2773578 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:57.643390 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d287d4-f76c-459a-a8cc-f8478412171f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:57.644271 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d287d4-f76c-459a-a8cc-f8478412171f admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:57.644709 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d287d4-f76c-459a-a8cc-f8478412171f admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:57.644963 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d287d4-f76c-459a-a8cc-f8478412171f admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:57.646369 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d287d4-f76c-459a-a8cc-f8478412171f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:57.647859 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d287d4-f76c-459a-a8cc-f8478412171f admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:57.672644 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1370/2754] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:29:57 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:29:57.677091 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa454c-c988-4d97-9b84-df7903eb02f4 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:57.687625 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa454c-c988-4d97-9b84-df7903eb02f4 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:57.692682 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6860bfe-a297-46db-8723-42cff7070188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:57.693283 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6860bfe-a297-46db-8723-42cff7070188 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:57.693547 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6860bfe-a297-46db-8723-42cff7070188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:57.693810 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6860bfe-a297-46db-8723-42cff7070188 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:57.694488 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6860bfe-a297-46db-8723-42cff7070188 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:57.696086 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6860bfe-a297-46db-8723-42cff7070188 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:57.705933 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1385/2755] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:29:57.709949 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d287d4-f76c-459a-a8cc-f8478412171f admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:57.721363 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d287d4-f76c-459a-a8cc-f8478412171f admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:57.726640 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c613e34a-d708-459b-80a4-9518608e554a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:57.727230 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c613e34a-d708-459b-80a4-9518608e554a admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:57.727477 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c613e34a-d708-459b-80a4-9518608e554a admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:57.727752 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c613e34a-d708-459b-80a4-9518608e554a admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:57.728389 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c613e34a-d708-459b-80a4-9518608e554a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:57.729996 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c613e34a-d708-459b-80a4-9518608e554a admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:57.745269 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1371/2756] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:29:57.750933 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6860bfe-a297-46db-8723-42cff7070188 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:57.761880 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6860bfe-a297-46db-8723-42cff7070188 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:57.768247 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfb5c25-ce26-4fd3-b534-e73d3fde0c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:57.768893 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb5c25-ce26-4fd3-b534-e73d3fde0c97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:57.769163 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb5c25-ce26-4fd3-b534-e73d3fde0c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:57.769425 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb5c25-ce26-4fd3-b534-e73d3fde0c97 admin admin] PATH_INFO: `/v3/projects/7ab71ae63f2c4ac59456507caf29652a/users/936b971d7604471ba7016253f245c4de/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:57.785734 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfb5c25-ce26-4fd3-b534-e73d3fde0c97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ab71ae63f2c4ac59456507caf29652a, user_id=936b971d7604471ba7016253f245c4de, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:57.788438 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfb5c25-ce26-4fd3-b534-e73d3fde0c97 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:57.803945 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1386/2757] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:29:57 2022] PUT /identity/v3/projects/7ab71ae63f2c4ac59456507caf29652a/users/936b971d7604471ba7016253f245c4de/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:29:57.814509 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053b8d2-7f04-4a5f-841c-b0494403649c None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:57.814790 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053b8d2-7f04-4a5f-841c-b0494403649c None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:57.815054 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053b8d2-7f04-4a5f-841c-b0494403649c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:57.843264 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-3053b8d2-7f04-4a5f-841c-b0494403649c None None] MFA Rules not processed for user `936b971d7604471ba7016253f245c4de`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:29:57.844486 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3053b8d2-7f04-4a5f-841c-b0494403649c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:57.913206 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1372/2758] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:29:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:29:57.925125 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfb5c25-ce26-4fd3-b534-e73d3fde0c97 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:29:57.927419 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ebfb5c25-ce26-4fd3-b534-e73d3fde0c97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:29:57.994069 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfb5c25-ce26-4fd3-b534-e73d3fde0c97 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:29:57.999612 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058168ba-04be-448b-91b0-0d1afd3be38c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:29:58.000215 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058168ba-04be-448b-91b0-0d1afd3be38c service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:29:58.000479 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058168ba-04be-448b-91b0-0d1afd3be38c service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:29:58.000762 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058168ba-04be-448b-91b0-0d1afd3be38c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:29:58.012364 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058168ba-04be-448b-91b0-0d1afd3be38c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:29:58.013823 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058168ba-04be-448b-91b0-0d1afd3be38c service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:29:58.023455 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1387/2759] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:29:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:30:00.601812 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3053b8d2-7f04-4a5f-841c-b0494403649c None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:30:00.660129 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:30:00.680304 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3053b8d2-7f04-4a5f-841c-b0494403649c None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:30:00.684976 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b182a460-807c-4be5-9835-1cc6e51a715e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:30:00.685618 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182a460-807c-4be5-9835-1cc6e51a715e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:30:00.685917 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182a460-807c-4be5-9835-1cc6e51a715e admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:30:00.686219 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182a460-807c-4be5-9835-1cc6e51a715e admin admin] PATH_INFO: `/v3/users/936b971d7604471ba7016253f245c4de` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:30:00.687354 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b182a460-807c-4be5-9835-1cc6e51a715e admin admin] RBAC: Authorizing `identity:delete_user(user_id=936b971d7604471ba7016253f245c4de)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:30:00.689293 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b182a460-807c-4be5-9835-1cc6e51a715e admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:30:00.744388 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-b182a460-807c-4be5-9835-1cc6e51a715e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '936b971d7604471ba7016253f245c4de'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:30:00.752370 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-b182a460-807c-4be5-9835-1cc6e51a715e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '936b971d7604471ba7016253f245c4de'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:30:00.753184 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-b182a460-807c-4be5-9835-1cc6e51a715e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '936b971d7604471ba7016253f245c4de'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:30:00.766336 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-b182a460-807c-4be5-9835-1cc6e51a715e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '936b971d7604471ba7016253f245c4de'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:30:00.767169 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-b182a460-807c-4be5-9835-1cc6e51a715e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '936b971d7604471ba7016253f245c4de'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:30:00.785422 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1373/2760] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:30:00 2022] DELETE /identity/v3/users/936b971d7604471ba7016253f245c4de => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:30:00.790145 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8422cfb-c809-42f6-b6e3-352906384799 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:30:00.790620 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8422cfb-c809-42f6-b6e3-352906384799 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:30:00.790720 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8422cfb-c809-42f6-b6e3-352906384799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:30:00.809367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f8422cfb-c809-42f6-b6e3-352906384799 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:30:00.810592 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f8422cfb-c809-42f6-b6e3-352906384799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:30:00.826764 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:30:00.872229 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1388/2761] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:30:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:30:00.882817 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b182a460-807c-4be5-9835-1cc6e51a715e admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:30:00.884432 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b182a460-807c-4be5-9835-1cc6e51a715e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:30:00.944519 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b182a460-807c-4be5-9835-1cc6e51a715e admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:30:00.948769 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f40a49-c453-4300-a1ae-b32e9f9d50d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:30:00.949381 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f40a49-c453-4300-a1ae-b32e9f9d50d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:30:00.949630 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f40a49-c453-4300-a1ae-b32e9f9d50d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:30:00.949896 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f40a49-c453-4300-a1ae-b32e9f9d50d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:30:00.960392 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f40a49-c453-4300-a1ae-b32e9f9d50d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:30:00.961841 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f40a49-c453-4300-a1ae-b32e9f9d50d9 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:30:00.975388 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1374/2762] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:30:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:30:01.527886 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8422cfb-c809-42f6-b6e3-352906384799 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:30:01.529501 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f8422cfb-c809-42f6-b6e3-352906384799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:30:01.542704 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8422cfb-c809-42f6-b6e3-352906384799 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:30:01.548075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e931be9-2827-479c-99d3-bf841dfe1264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:30:01.548789 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e931be9-2827-479c-99d3-bf841dfe1264 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:30:01.549131 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e931be9-2827-479c-99d3-bf841dfe1264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:30:01.549510 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e931be9-2827-479c-99d3-bf841dfe1264 admin admin] PATH_INFO: `/v3/projects/7ab71ae63f2c4ac59456507caf29652a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:30:01.550810 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e931be9-2827-479c-99d3-bf841dfe1264 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ab71ae63f2c4ac59456507caf29652a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:30:01.552777 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e931be9-2827-479c-99d3-bf841dfe1264 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:30:01.602550 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9e931be9-2827-479c-99d3-bf841dfe1264 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ab71ae63f2c4ac59456507caf29652a', 'request_id': 'req-9e931be9-2827-479c-99d3-bf841dfe1264'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:30:01.608604 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9e931be9-2827-479c-99d3-bf841dfe1264 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ab71ae63f2c4ac59456507caf29652a', 'request_id': 'req-9e931be9-2827-479c-99d3-bf841dfe1264'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:30:01.617146 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9e931be9-2827-479c-99d3-bf841dfe1264 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ab71ae63f2c4ac59456507caf29652a', 'request_id': 'req-9e931be9-2827-479c-99d3-bf841dfe1264'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:30:01.627453 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9e931be9-2827-479c-99d3-bf841dfe1264 admin admin] The token cache is being invalidate because project 7ab71ae63f2c4ac59456507caf29652a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:30:01.627738 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9e931be9-2827-479c-99d3-bf841dfe1264 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:30:01.628384 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9e931be9-2827-479c-99d3-bf841dfe1264 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:30:01.629894 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1389/2763] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:30:01 2022] DELETE /identity/v3/projects/7ab71ae63f2c4ac59456507caf29652a => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:30:10.967543 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=60,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:30:11.624767 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=44,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:30:25.520632 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f40a49-c453-4300-a1ae-b32e9f9d50d9 service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:30:25.522216 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-29f40a49-c453-4300-a1ae-b32e9f9d50d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:30:25.601639 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f40a49-c453-4300-a1ae-b32e9f9d50d9 service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:30:25.672960 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ffd56-c905-4f19-ac6e-03770db89496 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:30:25.673591 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ffd56-c905-4f19-ac6e-03770db89496 service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:30:25.673882 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ffd56-c905-4f19-ac6e-03770db89496 service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:30:25.674240 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ffd56-c905-4f19-ac6e-03770db89496 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:30:25.676228 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-693ffd56-c905-4f19-ac6e-03770db89496 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:30:25.690155 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693ffd56-c905-4f19-ac6e-03770db89496 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:30:25.691789 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693ffd56-c905-4f19-ac6e-03770db89496 service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:30:25.702698 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1375/2764] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:30:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:30:35.698042 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=62 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:30:37.317034 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401d1e0-cf81-487e-9a56-ae75b89fcd33 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:30:37.317569 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401d1e0-cf81-487e-9a56-ae75b89fcd33 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:30:37.317569 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401d1e0-cf81-487e-9a56-ae75b89fcd33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:30:37.339498 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-3401d1e0-cf81-487e-9a56-ae75b89fcd33 None None] MFA Rules not processed for user `695bbf71deee465985055e79dc8e105e`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:30:37.340700 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3401d1e0-cf81-487e-9a56-ae75b89fcd33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:30:37.397196 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1390/2765] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:30:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:30:37.952698 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ffd56-c905-4f19-ac6e-03770db89496 service nova] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:30:37.954279 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-693ffd56-c905-4f19-ac6e-03770db89496 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:30:38.033059 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ffd56-c905-4f19-ac6e-03770db89496 service nova] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:30:38.038352 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb067cc-93fb-45f7-922c-001724896523 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd4387035099416789009f0ecb8ad282', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:30:38.039141 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb067cc-93fb-45f7-922c-001724896523 service placement] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:30:38.040463 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb067cc-93fb-45f7-922c-001724896523 service placement] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:30:38.040463 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb067cc-93fb-45f7-922c-001724896523 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:30:38.044332 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-7eb067cc-93fb-45f7-922c-001724896523 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:30:38.056944 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb067cc-93fb-45f7-922c-001724896523 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:30:38.058884 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb067cc-93fb-45f7-922c-001724896523 service placement] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:30:38.070451 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1376/2766] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:30:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:30:46.958206 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3401d1e0-cf81-487e-9a56-ae75b89fcd33 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:30:46.959880 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3401d1e0-cf81-487e-9a56-ae75b89fcd33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:30:46.977704 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3401d1e0-cf81-487e-9a56-ae75b89fcd33 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:30:46.983984 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d29c90-ec28-4999-a3a9-6cc9c7222e8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:30:46.984605 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d29c90-ec28-4999-a3a9-6cc9c7222e8e service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:30:46.984877 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d29c90-ec28-4999-a3a9-6cc9c7222e8e service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:30:46.985139 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d29c90-ec28-4999-a3a9-6cc9c7222e8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:30:46.986934 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d5d29c90-ec28-4999-a3a9-6cc9c7222e8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:30:46.999354 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d29c90-ec28-4999-a3a9-6cc9c7222e8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:30:47.001075 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d29c90-ec28-4999-a3a9-6cc9c7222e8e service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:30:47.012102 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1391/2767] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:30:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:30:48.065626 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=32 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:30:57.007493 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=31 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:31:20.449151 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f809704-a7b7-44d6-a1a9-b5c224a99e55 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:31:20.449975 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f809704-a7b7-44d6-a1a9-b5c224a99e55 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:31:20.449975 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f809704-a7b7-44d6-a1a9-b5c224a99e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:31:20.469726 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-8f809704-a7b7-44d6-a1a9-b5c224a99e55 None None] MFA Rules not processed for user `b0f728c7e4074084be1a433ae14879b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:31:20.471037 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-8f809704-a7b7-44d6-a1a9-b5c224a99e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:31:20.527174 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1377/2768] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:31:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:31:20.848701 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d29c90-ec28-4999-a3a9-6cc9c7222e8e service nova] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:31:20.850304 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d5d29c90-ec28-4999-a3a9-6cc9c7222e8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:31:20.910338 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d29c90-ec28-4999-a3a9-6cc9c7222e8e service nova] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:31:20.914323 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e868a226-617b-49e4-8c5b-19e79e6bd4d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:31:20.914942 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e868a226-617b-49e4-8c5b-19e79e6bd4d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:31:20.915218 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e868a226-617b-49e4-8c5b-19e79e6bd4d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:31:20.915481 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e868a226-617b-49e4-8c5b-19e79e6bd4d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:31:20.917310 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e868a226-617b-49e4-8c5b-19e79e6bd4d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:31:20.927857 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e868a226-617b-49e4-8c5b-19e79e6bd4d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:31:20.929288 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e868a226-617b-49e4-8c5b-19e79e6bd4d0 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:31:20.937313 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1392/2769] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:31:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:31:30.519658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=25 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:31:30.933863 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=26 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:31:43.919309 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f809704-a7b7-44d6-a1a9-b5c224a99e55 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:31:43.920794 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-8f809704-a7b7-44d6-a1a9-b5c224a99e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:31:43.936101 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f809704-a7b7-44d6-a1a9-b5c224a99e55 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:31:43.941884 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea0567e-0730-4442-a81a-907ea148d500 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd4387035099416789009f0ecb8ad282', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:31:43.942542 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0567e-0730-4442-a81a-907ea148d500 service placement] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:31:43.942875 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0567e-0730-4442-a81a-907ea148d500 service placement] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:31:43.943156 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0567e-0730-4442-a81a-907ea148d500 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:31:43.944945 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-4ea0567e-0730-4442-a81a-907ea148d500 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:31:43.957028 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea0567e-0730-4442-a81a-907ea148d500 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:31:43.958665 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea0567e-0730-4442-a81a-907ea148d500 service placement] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:31:43.970167 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1378/2770] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:31:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:31:43.984836 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e868a226-617b-49e4-8c5b-19e79e6bd4d0 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:31:43.986322 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e868a226-617b-49e4-8c5b-19e79e6bd4d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:31:44.055972 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e868a226-617b-49e4-8c5b-19e79e6bd4d0 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:31:44.060934 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a70b27-b18a-4264-b992-a428d3f21671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:31:44.061567 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a70b27-b18a-4264-b992-a428d3f21671 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:31:44.061850 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a70b27-b18a-4264-b992-a428d3f21671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:31:44.062145 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a70b27-b18a-4264-b992-a428d3f21671 admin admin] PATH_INFO: `/v3/users/695bbf71deee465985055e79dc8e105e` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:31:44.063324 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a70b27-b18a-4264-b992-a428d3f21671 admin admin] RBAC: Authorizing `identity:delete_user(user_id=695bbf71deee465985055e79dc8e105e)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:31:44.065243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a70b27-b18a-4264-b992-a428d3f21671 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:31:44.116860 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-19a70b27-b18a-4264-b992-a428d3f21671 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '695bbf71deee465985055e79dc8e105e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:31:44.124171 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-19a70b27-b18a-4264-b992-a428d3f21671 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '695bbf71deee465985055e79dc8e105e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:31:44.131984 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-19a70b27-b18a-4264-b992-a428d3f21671 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '695bbf71deee465985055e79dc8e105e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:31:44.132654 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-19a70b27-b18a-4264-b992-a428d3f21671 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '695bbf71deee465985055e79dc8e105e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:31:44.144327 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-19a70b27-b18a-4264-b992-a428d3f21671 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '695bbf71deee465985055e79dc8e105e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:31:44.150508 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1393/2771] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:31:43 2022] DELETE /identity/v3/users/695bbf71deee465985055e79dc8e105e => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:31:44.325965 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea0567e-0730-4442-a81a-907ea148d500 service placement] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:31:44.338684 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea0567e-0730-4442-a81a-907ea148d500 service placement] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:31:44.395877 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ccaf6-c7f6-416d-924f-47c2d6133e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:31:44.396533 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ccaf6-c7f6-416d-924f-47c2d6133e8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:31:44.396805 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ccaf6-c7f6-416d-924f-47c2d6133e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:31:44.397067 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ccaf6-c7f6-416d-924f-47c2d6133e8f admin admin] PATH_INFO: `/v3/projects/25348a2fd6ba43bbb3cf5e8126cd8a67` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:31:44.398206 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ccaf6-c7f6-416d-924f-47c2d6133e8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=25348a2fd6ba43bbb3cf5e8126cd8a67)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:31:44.400124 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ccaf6-c7f6-416d-924f-47c2d6133e8f admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:31:44.447349 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-d66ccaf6-c7f6-416d-924f-47c2d6133e8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25348a2fd6ba43bbb3cf5e8126cd8a67', 'request_id': 'req-d66ccaf6-c7f6-416d-924f-47c2d6133e8f'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:31:44.453595 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-d66ccaf6-c7f6-416d-924f-47c2d6133e8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25348a2fd6ba43bbb3cf5e8126cd8a67', 'request_id': 'req-d66ccaf6-c7f6-416d-924f-47c2d6133e8f'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:31:44.460035 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-d66ccaf6-c7f6-416d-924f-47c2d6133e8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25348a2fd6ba43bbb3cf5e8126cd8a67', 'request_id': 'req-d66ccaf6-c7f6-416d-924f-47c2d6133e8f'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:31:44.476394 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-d66ccaf6-c7f6-416d-924f-47c2d6133e8f admin admin] The token cache is being invalidate because project 25348a2fd6ba43bbb3cf5e8126cd8a67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:31:44.476680 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-d66ccaf6-c7f6-416d-924f-47c2d6133e8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:31:44.477432 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-d66ccaf6-c7f6-416d-924f-47c2d6133e8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:31:44.479058 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1379/2772] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:31:44 2022] DELETE /identity/v3/projects/25348a2fd6ba43bbb3cf5e8126cd8a67 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:31:44.484290 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a40e2-edf1-4896-822a-6699b589be2f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:31:44.484578 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a40e2-edf1-4896-822a-6699b589be2f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:31:44.484848 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a40e2-edf1-4896-822a-6699b589be2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:31:44.505662 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-bd3a40e2-edf1-4896-822a-6699b589be2f None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:31:44.506949 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-bd3a40e2-edf1-4896-822a-6699b589be2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:31:44.568121 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1394/2773] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:31:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:31:44.579625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86776280-1275-4e56-9693-cf1b484dca31 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:31:44.580053 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86776280-1275-4e56-9693-cf1b484dca31 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:31:44.580487 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86776280-1275-4e56-9693-cf1b484dca31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:31:44.604723 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-86776280-1275-4e56-9693-cf1b484dca31 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:31:44.605957 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-86776280-1275-4e56-9693-cf1b484dca31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:31:44.615713 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1380/2774] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:31:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:31:44.621041 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a40e2-edf1-4896-822a-6699b589be2f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:31:44.633356 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a40e2-edf1-4896-822a-6699b589be2f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:31:44.639181 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a442e82-01ab-470f-8428-9ba59320abc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:31:44.639880 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a442e82-01ab-470f-8428-9ba59320abc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:31:44.640235 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a442e82-01ab-470f-8428-9ba59320abc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:31:44.640525 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a442e82-01ab-470f-8428-9ba59320abc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:31:44.641233 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a442e82-01ab-470f-8428-9ba59320abc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:31:44.641543 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a442e82-01ab-470f-8428-9ba59320abc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:31:44.643179 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a442e82-01ab-470f-8428-9ba59320abc3 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:31:44.656218 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1395/2775] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:31:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:31:44.661065 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86776280-1275-4e56-9693-cf1b484dca31 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:31:44.671000 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86776280-1275-4e56-9693-cf1b484dca31 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:31:44.675847 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d0eb3-3213-445f-869f-f3bd26959afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:31:44.676448 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d0eb3-3213-445f-869f-f3bd26959afe admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:31:44.676729 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d0eb3-3213-445f-869f-f3bd26959afe admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:31:44.677001 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d0eb3-3213-445f-869f-f3bd26959afe admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:31:44.677668 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13d0eb3-3213-445f-869f-f3bd26959afe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:31:44.679630 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13d0eb3-3213-445f-869f-f3bd26959afe admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:31:44.713330 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1381/2776] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:31:44 2022] POST /identity/v3/projects => generated 391 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:31:44.717708 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a442e82-01ab-470f-8428-9ba59320abc3 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:31:44.728285 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a442e82-01ab-470f-8428-9ba59320abc3 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:31:44.731909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:31:44.786026 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54a7ecb-6da4-4088-a184-9ab6b45435d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:31:44.786703 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a7ecb-6da4-4088-a184-9ab6b45435d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:31:44.786987 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a7ecb-6da4-4088-a184-9ab6b45435d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:31:44.787251 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a7ecb-6da4-4088-a184-9ab6b45435d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:31:44.787915 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54a7ecb-6da4-4088-a184-9ab6b45435d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:31:44.789828 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54a7ecb-6da4-4088-a184-9ab6b45435d8 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:31:44.813355 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1396/2777] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:31:44 2022] POST /identity/v3/users => generated 296 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:31:44.818628 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d0eb3-3213-445f-869f-f3bd26959afe admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:31:44.828814 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d0eb3-3213-445f-869f-f3bd26959afe admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:31:44.833640 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b4e3c3-540d-4038-af6f-43bd6cad42d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:31:44.834259 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b4e3c3-540d-4038-af6f-43bd6cad42d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:31:44.834540 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b4e3c3-540d-4038-af6f-43bd6cad42d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:31:44.834814 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b4e3c3-540d-4038-af6f-43bd6cad42d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:31:44.835465 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b4e3c3-540d-4038-af6f-43bd6cad42d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:31:44.837119 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b4e3c3-540d-4038-af6f-43bd6cad42d6 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:31:44.847043 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1382/2778] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:31:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:31:44.851790 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54a7ecb-6da4-4088-a184-9ab6b45435d8 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:31:44.862153 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54a7ecb-6da4-4088-a184-9ab6b45435d8 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:31:44.867592 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92fc87-444b-420f-a475-c522bb08bf32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:31:44.868189 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92fc87-444b-420f-a475-c522bb08bf32 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:31:44.868463 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92fc87-444b-420f-a475-c522bb08bf32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:31:44.868728 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92fc87-444b-420f-a475-c522bb08bf32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:31:44.869367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b92fc87-444b-420f-a475-c522bb08bf32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:31:44.871016 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b92fc87-444b-420f-a475-c522bb08bf32 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:31:44.881797 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1397/2779] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:31:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:31:44.886665 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b4e3c3-540d-4038-af6f-43bd6cad42d6 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:31:44.899409 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b4e3c3-540d-4038-af6f-43bd6cad42d6 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:31:44.906909 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca65bcd-8450-45cd-800e-86f6e5d49c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:31:44.907932 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca65bcd-8450-45cd-800e-86f6e5d49c64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:31:44.908275 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca65bcd-8450-45cd-800e-86f6e5d49c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:31:44.908608 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca65bcd-8450-45cd-800e-86f6e5d49c64 admin admin] PATH_INFO: `/v3/projects/7031f4ba713a42d6adb34220eab0f315/users/49183a1ee5424ad0bc118eca3a85e4c2/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:31:44.924138 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca65bcd-8450-45cd-800e-86f6e5d49c64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7031f4ba713a42d6adb34220eab0f315, user_id=49183a1ee5424ad0bc118eca3a85e4c2, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:31:44.926850 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca65bcd-8450-45cd-800e-86f6e5d49c64 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:31:44.944192 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1383/2780] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:31:44 2022] PUT /identity/v3/projects/7031f4ba713a42d6adb34220eab0f315/users/49183a1ee5424ad0bc118eca3a85e4c2/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:31:44.955696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2effce-5647-4ff7-bd20-6dfcda5f988a None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:31:44.956001 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2effce-5647-4ff7-bd20-6dfcda5f988a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:31:44.956268 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2effce-5647-4ff7-bd20-6dfcda5f988a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:31:44.985327 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-2f2effce-5647-4ff7-bd20-6dfcda5f988a None None] MFA Rules not processed for user `49183a1ee5424ad0bc118eca3a85e4c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:31:44.986482 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2f2effce-5647-4ff7-bd20-6dfcda5f988a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:31:45.053840 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1398/2781] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:31:44 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:31:45.064822 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca65bcd-8450-45cd-800e-86f6e5d49c64 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:31:45.066614 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-3ca65bcd-8450-45cd-800e-86f6e5d49c64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:31:45.081131 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:31:45.126593 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca65bcd-8450-45cd-800e-86f6e5d49c64 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:31:45.130280 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb6ef19-261e-4d4d-a861-53c67458901f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:31:45.130886 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6ef19-261e-4d4d-a861-53c67458901f service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:31:45.131162 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6ef19-261e-4d4d-a861-53c67458901f service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:31:45.131431 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6ef19-261e-4d4d-a861-53c67458901f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:31:45.140239 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb6ef19-261e-4d4d-a861-53c67458901f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:31:45.142190 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb6ef19-261e-4d4d-a861-53c67458901f service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:31:45.152116 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1384/2782] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:31:45 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:31:55.051086 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=63,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:31:55.147485 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=20 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:32:06.736771 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2effce-5647-4ff7-bd20-6dfcda5f988a None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:32:06.828667 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2effce-5647-4ff7-bd20-6dfcda5f988a None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:32:06.835019 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969668a1-75ca-49ca-bc5d-5b6f6acb7b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:32:06.835663 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969668a1-75ca-49ca-bc5d-5b6f6acb7b5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:32:06.835934 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969668a1-75ca-49ca-bc5d-5b6f6acb7b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:32:06.836196 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969668a1-75ca-49ca-bc5d-5b6f6acb7b5a admin admin] PATH_INFO: `/v3/users/49183a1ee5424ad0bc118eca3a85e4c2` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:32:06.837468 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969668a1-75ca-49ca-bc5d-5b6f6acb7b5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=49183a1ee5424ad0bc118eca3a85e4c2)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:32:06.839444 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969668a1-75ca-49ca-bc5d-5b6f6acb7b5a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:32:06.889801 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-969668a1-75ca-49ca-bc5d-5b6f6acb7b5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49183a1ee5424ad0bc118eca3a85e4c2'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:32:06.896741 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-969668a1-75ca-49ca-bc5d-5b6f6acb7b5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49183a1ee5424ad0bc118eca3a85e4c2'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:32:06.905726 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-969668a1-75ca-49ca-bc5d-5b6f6acb7b5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49183a1ee5424ad0bc118eca3a85e4c2'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:32:06.906452 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-969668a1-75ca-49ca-bc5d-5b6f6acb7b5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49183a1ee5424ad0bc118eca3a85e4c2'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:32:06.918921 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-969668a1-75ca-49ca-bc5d-5b6f6acb7b5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49183a1ee5424ad0bc118eca3a85e4c2'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:32:06.924909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1399/2783] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:32:06 2022] DELETE /identity/v3/users/49183a1ee5424ad0bc118eca3a85e4c2 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:32:06.929561 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62346a28-f905-4e76-b2ae-7dedccd99d04 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:32:06.930107 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62346a28-f905-4e76-b2ae-7dedccd99d04 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:32:06.930215 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62346a28-f905-4e76-b2ae-7dedccd99d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:32:06.954703 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-62346a28-f905-4e76-b2ae-7dedccd99d04 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:32:06.956021 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-62346a28-f905-4e76-b2ae-7dedccd99d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:32:07.029770 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1385/2784] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:32:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:32:07.040060 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969668a1-75ca-49ca-bc5d-5b6f6acb7b5a admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:32:07.041535 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-969668a1-75ca-49ca-bc5d-5b6f6acb7b5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:32:07.101554 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969668a1-75ca-49ca-bc5d-5b6f6acb7b5a admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:32:07.105769 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ec4c9-85f8-47fa-b93c-63ffbe454a82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:32:07.106408 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ec4c9-85f8-47fa-b93c-63ffbe454a82 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:32:07.106680 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ec4c9-85f8-47fa-b93c-63ffbe454a82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:32:07.106947 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ec4c9-85f8-47fa-b93c-63ffbe454a82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:32:07.117122 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ec4c9-85f8-47fa-b93c-63ffbe454a82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:32:07.118711 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ec4c9-85f8-47fa-b93c-63ffbe454a82 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:32:07.131047 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1400/2785] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:32:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:32:07.409910 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62346a28-f905-4e76-b2ae-7dedccd99d04 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:32:07.411886 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-62346a28-f905-4e76-b2ae-7dedccd99d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:32:07.426150 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62346a28-f905-4e76-b2ae-7dedccd99d04 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:32:07.433256 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae66a20d-5ab7-41bf-8c61-f00f3d5cdd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:32:07.433977 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66a20d-5ab7-41bf-8c61-f00f3d5cdd92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:32:07.434283 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66a20d-5ab7-41bf-8c61-f00f3d5cdd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:32:07.434634 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66a20d-5ab7-41bf-8c61-f00f3d5cdd92 admin admin] PATH_INFO: `/v3/projects/7031f4ba713a42d6adb34220eab0f315` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:32:07.436248 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae66a20d-5ab7-41bf-8c61-f00f3d5cdd92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7031f4ba713a42d6adb34220eab0f315)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:32:07.438716 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae66a20d-5ab7-41bf-8c61-f00f3d5cdd92 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:32:07.491878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ae66a20d-5ab7-41bf-8c61-f00f3d5cdd92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7031f4ba713a42d6adb34220eab0f315', 'request_id': 'req-ae66a20d-5ab7-41bf-8c61-f00f3d5cdd92'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:32:07.498096 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ae66a20d-5ab7-41bf-8c61-f00f3d5cdd92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7031f4ba713a42d6adb34220eab0f315', 'request_id': 'req-ae66a20d-5ab7-41bf-8c61-f00f3d5cdd92'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:32:07.503783 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ae66a20d-5ab7-41bf-8c61-f00f3d5cdd92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7031f4ba713a42d6adb34220eab0f315', 'request_id': 'req-ae66a20d-5ab7-41bf-8c61-f00f3d5cdd92'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:32:07.517460 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ae66a20d-5ab7-41bf-8c61-f00f3d5cdd92 admin admin] The token cache is being invalidate because project 7031f4ba713a42d6adb34220eab0f315 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:32:07.517743 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ae66a20d-5ab7-41bf-8c61-f00f3d5cdd92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:32:07.518472 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-ae66a20d-5ab7-41bf-8c61-f00f3d5cdd92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:32:07.520057 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1386/2786] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:32:07 2022] DELETE /identity/v3/projects/7031f4ba713a42d6adb34220eab0f315 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:32:17.124486 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:32:17.514287 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:32:49.563692 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ec4c9-85f8-47fa-b93c-63ffbe454a82 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:32:49.565411 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e58ec4c9-85f8-47fa-b93c-63ffbe454a82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:32:49.638143 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ec4c9-85f8-47fa-b93c-63ffbe454a82 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:32:49.705558 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837eb46-e5ea-4e88-bf25-b9d192ec00dc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd4387035099416789009f0ecb8ad282', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:32:49.706261 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837eb46-e5ea-4e88-bf25-b9d192ec00dc service placement] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:32:49.706548 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837eb46-e5ea-4e88-bf25-b9d192ec00dc service placement] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:32:49.706810 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837eb46-e5ea-4e88-bf25-b9d192ec00dc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:32:49.708644 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b837eb46-e5ea-4e88-bf25-b9d192ec00dc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:32:49.719650 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b837eb46-e5ea-4e88-bf25-b9d192ec00dc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:32:49.721226 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b837eb46-e5ea-4e88-bf25-b9d192ec00dc service placement] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:32:49.731874 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1401/2787] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:32:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:32:54.466805 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae66a20d-5ab7-41bf-8c61-f00f3d5cdd92 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:32:54.468541 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-ae66a20d-5ab7-41bf-8c61-f00f3d5cdd92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:32:54.545878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae66a20d-5ab7-41bf-8c61-f00f3d5cdd92 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:32:54.552240 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee5704d-1d4d-4d3b-8d0f-f8f8051429d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:32:54.552843 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee5704d-1d4d-4d3b-8d0f-f8f8051429d8 service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:32:54.553120 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee5704d-1d4d-4d3b-8d0f-f8f8051429d8 service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:32:54.553386 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee5704d-1d4d-4d3b-8d0f-f8f8051429d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:32:54.555300 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-1ee5704d-1d4d-4d3b-8d0f-f8f8051429d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:32:54.622864 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee5704d-1d4d-4d3b-8d0f-f8f8051429d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:32:54.624550 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee5704d-1d4d-4d3b-8d0f-f8f8051429d8 service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:32:54.634302 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1387/2788] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:32:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:32:59.727238 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=58 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:33:04.629913 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=56 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:33:11.230564 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6c8571-353c-44f8-af6a-5037cac5df70 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:33:11.231011 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6c8571-353c-44f8-af6a-5037cac5df70 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:33:11.231133 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6c8571-353c-44f8-af6a-5037cac5df70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:33:11.250598 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-6b6c8571-353c-44f8-af6a-5037cac5df70 None None] MFA Rules not processed for user `b0f728c7e4074084be1a433ae14879b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:33:11.251866 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6b6c8571-353c-44f8-af6a-5037cac5df70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:33:11.259714 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1402/2789] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:33:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:33:21.244789 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:34:34.406486 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee5704d-1d4d-4d3b-8d0f-f8f8051429d8 service nova] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:34:34.408189 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-1ee5704d-1d4d-4d3b-8d0f-f8f8051429d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:34:34.468889 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee5704d-1d4d-4d3b-8d0f-f8f8051429d8 service nova] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:34:34.472691 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c923a-7e81-47dc-8ff1-36d0912bb481 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:34:34.473287 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c923a-7e81-47dc-8ff1-36d0912bb481 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:34:34.473560 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c923a-7e81-47dc-8ff1-36d0912bb481 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:34:34.473825 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c923a-7e81-47dc-8ff1-36d0912bb481 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:34:34.475866 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-577c923a-7e81-47dc-8ff1-36d0912bb481 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:34:34.491142 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577c923a-7e81-47dc-8ff1-36d0912bb481 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:34:34.492572 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577c923a-7e81-47dc-8ff1-36d0912bb481 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:34:34.504108 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1388/2790] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:34:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:34:42.515130 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6c8571-353c-44f8-af6a-5037cac5df70 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:34:42.529905 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6c8571-353c-44f8-af6a-5037cac5df70 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:34:42.534335 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f6725c-71e9-474c-b44e-c4d9cca27ee9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:34:42.534955 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6725c-71e9-474c-b44e-c4d9cca27ee9 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:34:42.535226 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6725c-71e9-474c-b44e-c4d9cca27ee9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:34:42.535488 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6725c-71e9-474c-b44e-c4d9cca27ee9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:34:42.537256 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e0f6725c-71e9-474c-b44e-c4d9cca27ee9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:34:42.550764 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f6725c-71e9-474c-b44e-c4d9cca27ee9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:34:42.552180 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f6725c-71e9-474c-b44e-c4d9cca27ee9 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:34:42.562967 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1403/2791] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:34:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 05:34:44.499158 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=26 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:34:51.924727 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c923a-7e81-47dc-8ff1-36d0912bb481 service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:34:51.926160 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-577c923a-7e81-47dc-8ff1-36d0912bb481 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:34:51.986598 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c923a-7e81-47dc-8ff1-36d0912bb481 service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:34:51.990966 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccef8cfd-da5a-4c46-94fc-a9b065b5e811 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:34:51.991568 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef8cfd-da5a-4c46-94fc-a9b065b5e811 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:34:51.991844 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef8cfd-da5a-4c46-94fc-a9b065b5e811 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:34:51.992111 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef8cfd-da5a-4c46-94fc-a9b065b5e811 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:34:51.994145 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-ccef8cfd-da5a-4c46-94fc-a9b065b5e811 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:34:52.059233 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccef8cfd-da5a-4c46-94fc-a9b065b5e811 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:34:52.060667 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccef8cfd-da5a-4c46-94fc-a9b065b5e811 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:34:52.070443 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1389/2792] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:34:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:34:52.558382 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=6 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:34:55.260240 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f6725c-71e9-474c-b44e-c4d9cca27ee9 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:34:55.261663 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e0f6725c-71e9-474c-b44e-c4d9cca27ee9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:34:55.280818 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f6725c-71e9-474c-b44e-c4d9cca27ee9 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:34:55.299012 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111066d3-6eb9-4e38-b046-52a512635f43 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd4387035099416789009f0ecb8ad282', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:34:55.299627 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111066d3-6eb9-4e38-b046-52a512635f43 service placement] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:34:55.299903 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111066d3-6eb9-4e38-b046-52a512635f43 service placement] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:34:55.300171 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111066d3-6eb9-4e38-b046-52a512635f43 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:34:55.307654 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-111066d3-6eb9-4e38-b046-52a512635f43 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:34:55.322677 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111066d3-6eb9-4e38-b046-52a512635f43 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:34:55.325066 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111066d3-6eb9-4e38-b046-52a512635f43 service placement] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:34:55.341656 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1404/2793] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:34:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:34:55.351810 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccef8cfd-da5a-4c46-94fc-a9b065b5e811 service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:34:55.354956 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-ccef8cfd-da5a-4c46-94fc-a9b065b5e811 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:34:55.369358 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccef8cfd-da5a-4c46-94fc-a9b065b5e811 service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:34:55.374725 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0460d709-8fb6-4c13-8626-cc284d806fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:34:55.375360 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0460d709-8fb6-4c13-8626-cc284d806fdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:34:55.375638 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0460d709-8fb6-4c13-8626-cc284d806fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:34:55.376002 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0460d709-8fb6-4c13-8626-cc284d806fdd admin admin] PATH_INFO: `/v3/users/b0f728c7e4074084be1a433ae14879b8` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:34:55.377083 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0460d709-8fb6-4c13-8626-cc284d806fdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0f728c7e4074084be1a433ae14879b8)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:34:55.379073 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0460d709-8fb6-4c13-8626-cc284d806fdd admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:34:55.427048 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0460d709-8fb6-4c13-8626-cc284d806fdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0f728c7e4074084be1a433ae14879b8'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:34:55.433914 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0460d709-8fb6-4c13-8626-cc284d806fdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0f728c7e4074084be1a433ae14879b8'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:34:55.434594 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0460d709-8fb6-4c13-8626-cc284d806fdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0f728c7e4074084be1a433ae14879b8'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:34:55.443560 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0460d709-8fb6-4c13-8626-cc284d806fdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0f728c7e4074084be1a433ae14879b8'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:34:55.444215 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-0460d709-8fb6-4c13-8626-cc284d806fdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0f728c7e4074084be1a433ae14879b8'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:34:55.462143 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1390/2794] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:34:55 2022] DELETE /identity/v3/users/b0f728c7e4074084be1a433ae14879b8 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:34:55.659154 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111066d3-6eb9-4e38-b046-52a512635f43 service placement] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:34:55.671980 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111066d3-6eb9-4e38-b046-52a512635f43 service placement] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:34:55.729218 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9657d-4e3a-491b-8c04-36650ea6a286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:34:55.729843 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9657d-4e3a-491b-8c04-36650ea6a286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:34:55.730148 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9657d-4e3a-491b-8c04-36650ea6a286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:34:55.730410 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9657d-4e3a-491b-8c04-36650ea6a286 admin admin] PATH_INFO: `/v3/projects/cb164f6efb1d4047b9a5ca77f2ff44a7` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:34:55.731561 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d9657d-4e3a-491b-8c04-36650ea6a286 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb164f6efb1d4047b9a5ca77f2ff44a7)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:34:55.733456 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d9657d-4e3a-491b-8c04-36650ea6a286 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:34:55.793159 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-93d9657d-4e3a-491b-8c04-36650ea6a286 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb164f6efb1d4047b9a5ca77f2ff44a7', 'request_id': 'req-93d9657d-4e3a-491b-8c04-36650ea6a286'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:34:55.801039 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-93d9657d-4e3a-491b-8c04-36650ea6a286 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb164f6efb1d4047b9a5ca77f2ff44a7', 'request_id': 'req-93d9657d-4e3a-491b-8c04-36650ea6a286'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:34:55.810725 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-93d9657d-4e3a-491b-8c04-36650ea6a286 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb164f6efb1d4047b9a5ca77f2ff44a7', 'request_id': 'req-93d9657d-4e3a-491b-8c04-36650ea6a286'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:34:55.821460 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-93d9657d-4e3a-491b-8c04-36650ea6a286 admin admin] The token cache is being invalidate because project cb164f6efb1d4047b9a5ca77f2ff44a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:34:55.821741 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-93d9657d-4e3a-491b-8c04-36650ea6a286 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:34:55.822441 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-93d9657d-4e3a-491b-8c04-36650ea6a286 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:34:55.824030 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1405/2795] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:34:55 2022] DELETE /identity/v3/projects/cb164f6efb1d4047b9a5ca77f2ff44a7 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:34:55.829163 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88611692-0a9c-4b97-823c-3bacb3ce2059 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:34:55.829448 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88611692-0a9c-4b97-823c-3bacb3ce2059 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:34:55.829723 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88611692-0a9c-4b97-823c-3bacb3ce2059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:34:55.850313 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-88611692-0a9c-4b97-823c-3bacb3ce2059 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:34:55.851520 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-88611692-0a9c-4b97-823c-3bacb3ce2059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:34:55.869394 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:34:55.916858 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1391/2796] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:34:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:34:55.925200 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7cf9b4-a0b9-4e56-96cf-c7bb59c067e6 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:34:55.925486 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7cf9b4-a0b9-4e56-96cf-c7bb59c067e6 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:34:55.925757 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7cf9b4-a0b9-4e56-96cf-c7bb59c067e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:34:55.946999 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-9f7cf9b4-a0b9-4e56-96cf-c7bb59c067e6 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:34:55.948237 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9f7cf9b4-a0b9-4e56-96cf-c7bb59c067e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:34:55.957719 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1406/2797] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:34:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:34:55.962206 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88611692-0a9c-4b97-823c-3bacb3ce2059 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:34:55.972585 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88611692-0a9c-4b97-823c-3bacb3ce2059 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:34:55.978846 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2375fca-f378-4d52-99a7-b376aa3994da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:34:55.979468 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2375fca-f378-4d52-99a7-b376aa3994da admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:34:55.979748 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2375fca-f378-4d52-99a7-b376aa3994da admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:34:55.980014 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2375fca-f378-4d52-99a7-b376aa3994da admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:34:55.980699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2375fca-f378-4d52-99a7-b376aa3994da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:34:55.981005 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2375fca-f378-4d52-99a7-b376aa3994da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:34:55.982721 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2375fca-f378-4d52-99a7-b376aa3994da admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:34:55.995162 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1392/2798] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:34:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:34:55.999463 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7cf9b4-a0b9-4e56-96cf-c7bb59c067e6 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:34:56.008937 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7cf9b4-a0b9-4e56-96cf-c7bb59c067e6 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:34:56.014065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea9a49e-7265-4bb3-b2e7-f5c7f8bc1c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:34:56.014673 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9a49e-7265-4bb3-b2e7-f5c7f8bc1c84 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:34:56.014948 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9a49e-7265-4bb3-b2e7-f5c7f8bc1c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:34:56.015214 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9a49e-7265-4bb3-b2e7-f5c7f8bc1c84 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:34:56.015839 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea9a49e-7265-4bb3-b2e7-f5c7f8bc1c84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:34:56.017828 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea9a49e-7265-4bb3-b2e7-f5c7f8bc1c84 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:34:56.047466 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1407/2799] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:34:55 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:34:56.051588 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2375fca-f378-4d52-99a7-b376aa3994da admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:34:56.062497 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2375fca-f378-4d52-99a7-b376aa3994da admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:34:56.119928 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc0281-924e-488d-9e0a-14debd592a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:34:56.120526 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc0281-924e-488d-9e0a-14debd592a55 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:34:56.120803 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc0281-924e-488d-9e0a-14debd592a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:34:56.121072 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc0281-924e-488d-9e0a-14debd592a55 admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:34:56.121703 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dc0281-924e-488d-9e0a-14debd592a55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:34:56.123687 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dc0281-924e-488d-9e0a-14debd592a55 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:34:56.147936 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1393/2800] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:34:56 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:34:56.152290 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea9a49e-7265-4bb3-b2e7-f5c7f8bc1c84 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:34:56.167021 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea9a49e-7265-4bb3-b2e7-f5c7f8bc1c84 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:34:56.172564 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a127b736-c147-4eea-9da9-eade9ed1c234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:34:56.173228 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127b736-c147-4eea-9da9-eade9ed1c234 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:34:56.173522 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127b736-c147-4eea-9da9-eade9ed1c234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:34:56.173784 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127b736-c147-4eea-9da9-eade9ed1c234 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:34:56.174468 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a127b736-c147-4eea-9da9-eade9ed1c234 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:34:56.176056 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a127b736-c147-4eea-9da9-eade9ed1c234 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:34:56.186935 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1408/2801] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:34:56.191337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc0281-924e-488d-9e0a-14debd592a55 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:34:56.202689 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc0281-924e-488d-9e0a-14debd592a55 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:34:56.207952 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466cb2ef-2ece-4ec3-a12a-fd7f2fd77267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:34:56.208561 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466cb2ef-2ece-4ec3-a12a-fd7f2fd77267 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:34:56.208838 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466cb2ef-2ece-4ec3-a12a-fd7f2fd77267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:34:56.209108 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466cb2ef-2ece-4ec3-a12a-fd7f2fd77267 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:34:56.209750 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466cb2ef-2ece-4ec3-a12a-fd7f2fd77267 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:34:56.211395 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466cb2ef-2ece-4ec3-a12a-fd7f2fd77267 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:34:56.223554 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1394/2802] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:34:56.227752 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a127b736-c147-4eea-9da9-eade9ed1c234 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:34:56.238648 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a127b736-c147-4eea-9da9-eade9ed1c234 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:34:56.244301 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23f25c7-dae3-445b-ba7d-cdc134fa9158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:34:56.244949 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f25c7-dae3-445b-ba7d-cdc134fa9158 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:34:56.245222 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f25c7-dae3-445b-ba7d-cdc134fa9158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:34:56.245488 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f25c7-dae3-445b-ba7d-cdc134fa9158 admin admin] PATH_INFO: `/v3/projects/3eab51577af34120abf5eaa9a8ce8399/users/8acb4c939dd747958ecc2676c9efeeaf/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:34:56.260734 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23f25c7-dae3-445b-ba7d-cdc134fa9158 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3eab51577af34120abf5eaa9a8ce8399, user_id=8acb4c939dd747958ecc2676c9efeeaf, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:34:56.263433 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23f25c7-dae3-445b-ba7d-cdc134fa9158 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:34:56.279474 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1409/2803] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:34:56 2022] PUT /identity/v3/projects/3eab51577af34120abf5eaa9a8ce8399/users/8acb4c939dd747958ecc2676c9efeeaf/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:34:56.289987 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4fefcc-1a24-40bf-b143-233d5eaeffb4 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:34:56.290301 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4fefcc-1a24-40bf-b143-233d5eaeffb4 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:34:56.290571 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4fefcc-1a24-40bf-b143-233d5eaeffb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:34:56.319838 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-2b4fefcc-1a24-40bf-b143-233d5eaeffb4 None None] MFA Rules not processed for user `8acb4c939dd747958ecc2676c9efeeaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:34:56.321038 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-2b4fefcc-1a24-40bf-b143-233d5eaeffb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:34:56.385819 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1395/2804] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:34:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:34:56.390691 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23f25c7-dae3-445b-ba7d-cdc134fa9158 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:34:56.404096 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:34:56.452358 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23f25c7-dae3-445b-ba7d-cdc134fa9158 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:34:56.457364 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd82028-88bd-45d9-9371-cb4f6610fabb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:34:56.457961 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd82028-88bd-45d9-9371-cb4f6610fabb admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:34:56.458262 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd82028-88bd-45d9-9371-cb4f6610fabb admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:34:56.458525 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd82028-88bd-45d9-9371-cb4f6610fabb admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:34:56.459140 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd82028-88bd-45d9-9371-cb4f6610fabb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:34:56.461125 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd82028-88bd-45d9-9371-cb4f6610fabb admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:34:56.492865 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1410/2805] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:34:56 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:34:56.497303 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4fefcc-1a24-40bf-b143-233d5eaeffb4 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:34:56.508851 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4fefcc-1a24-40bf-b143-233d5eaeffb4 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:34:56.521419 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:34:56.566993 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c657caf-2675-4f15-93e9-a69fd6fbbebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:34:56.567714 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c657caf-2675-4f15-93e9-a69fd6fbbebc admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:34:56.567992 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c657caf-2675-4f15-93e9-a69fd6fbbebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:34:56.568260 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c657caf-2675-4f15-93e9-a69fd6fbbebc admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:34:56.568893 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c657caf-2675-4f15-93e9-a69fd6fbbebc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:34:56.570935 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c657caf-2675-4f15-93e9-a69fd6fbbebc admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:34:56.593308 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1396/2806] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:34:56 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:34:56.598231 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd82028-88bd-45d9-9371-cb4f6610fabb admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:34:56.609820 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd82028-88bd-45d9-9371-cb4f6610fabb admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:34:56.615316 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ef33ce-0be0-41dc-8db1-a44b4d86a3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:34:56.615920 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef33ce-0be0-41dc-8db1-a44b4d86a3ef admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:34:56.616200 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef33ce-0be0-41dc-8db1-a44b4d86a3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:34:56.616470 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef33ce-0be0-41dc-8db1-a44b4d86a3ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:34:56.617120 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ef33ce-0be0-41dc-8db1-a44b4d86a3ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:34:56.618783 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ef33ce-0be0-41dc-8db1-a44b4d86a3ef admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:34:56.629634 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1411/2807] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:34:56.635479 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c657caf-2675-4f15-93e9-a69fd6fbbebc admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:34:56.646617 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c657caf-2675-4f15-93e9-a69fd6fbbebc admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:34:56.651642 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e11ccae-31d9-444e-91e5-6a5e095b5413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:34:56.652287 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e11ccae-31d9-444e-91e5-6a5e095b5413 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:34:56.652564 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e11ccae-31d9-444e-91e5-6a5e095b5413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:34:56.652836 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e11ccae-31d9-444e-91e5-6a5e095b5413 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:34:56.653500 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e11ccae-31d9-444e-91e5-6a5e095b5413 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:34:56.655181 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e11ccae-31d9-444e-91e5-6a5e095b5413 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:34:56.665981 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1397/2808] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:34:56.670837 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ef33ce-0be0-41dc-8db1-a44b4d86a3ef admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:34:56.681476 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ef33ce-0be0-41dc-8db1-a44b4d86a3ef admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:34:56.686355 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5d5eab-48fe-4fad-b1f4-d5fec8a08cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:34:56.687007 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5d5eab-48fe-4fad-b1f4-d5fec8a08cfc admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:34:56.687278 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5d5eab-48fe-4fad-b1f4-d5fec8a08cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:34:56.687538 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5d5eab-48fe-4fad-b1f4-d5fec8a08cfc admin admin] PATH_INFO: `/v3/projects/dcf1ad1aa60847cf9a74d4bd97e03bf5/users/2ec6494b595e4d6cae7a4f1fb558a970/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:34:56.703343 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5d5eab-48fe-4fad-b1f4-d5fec8a08cfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcf1ad1aa60847cf9a74d4bd97e03bf5, user_id=2ec6494b595e4d6cae7a4f1fb558a970, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:34:56.706121 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5d5eab-48fe-4fad-b1f4-d5fec8a08cfc admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:34:56.721526 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1412/2809] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:34:56 2022] PUT /identity/v3/projects/dcf1ad1aa60847cf9a74d4bd97e03bf5/users/2ec6494b595e4d6cae7a4f1fb558a970/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:34:56.787199 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023d463-e56a-4fc7-bced-55b71211cae3 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:34:56.787514 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023d463-e56a-4fc7-bced-55b71211cae3 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:34:56.787760 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023d463-e56a-4fc7-bced-55b71211cae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:34:56.817680 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-8023d463-e56a-4fc7-bced-55b71211cae3 None None] MFA Rules not processed for user `2ec6494b595e4d6cae7a4f1fb558a970`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:34:56.818983 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-8023d463-e56a-4fc7-bced-55b71211cae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:34:56.886213 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1398/2810] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:34:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:34:56.892916 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5d5eab-48fe-4fad-b1f4-d5fec8a08cfc admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:34:56.965840 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5d5eab-48fe-4fad-b1f4-d5fec8a08cfc admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:34:56.973961 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecba2b62-b827-4c60-93d5-5495da56c4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:34:56.974916 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecba2b62-b827-4c60-93d5-5495da56c4dc admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:34:56.975353 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecba2b62-b827-4c60-93d5-5495da56c4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:34:56.975750 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecba2b62-b827-4c60-93d5-5495da56c4dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:34:56.976767 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecba2b62-b827-4c60-93d5-5495da56c4dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:34:56.980423 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecba2b62-b827-4c60-93d5-5495da56c4dc admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:34:57.018909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1413/2811] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:34:56 2022] POST /identity/v3/projects => generated 387 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:34:57.023920 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8023d463-e56a-4fc7-bced-55b71211cae3 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:34:57.036809 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8023d463-e56a-4fc7-bced-55b71211cae3 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:34:57.093855 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f79e6ed-fde2-47f8-a207-d047b9d512ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:34:57.094491 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79e6ed-fde2-47f8-a207-d047b9d512ec admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:34:57.094807 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79e6ed-fde2-47f8-a207-d047b9d512ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:34:57.095082 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79e6ed-fde2-47f8-a207-d047b9d512ec admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:34:57.095718 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f79e6ed-fde2-47f8-a207-d047b9d512ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:34:57.097653 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f79e6ed-fde2-47f8-a207-d047b9d512ec admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:34:57.120362 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1399/2812] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:34:57 2022] POST /identity/v3/users => generated 300 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:34:57.124723 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecba2b62-b827-4c60-93d5-5495da56c4dc admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:34:57.135685 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecba2b62-b827-4c60-93d5-5495da56c4dc admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:34:57.140419 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40666bf7-7907-4f54-8ba0-fa2369d4e805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:34:57.140962 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40666bf7-7907-4f54-8ba0-fa2369d4e805 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:34:57.141243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40666bf7-7907-4f54-8ba0-fa2369d4e805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:34:57.141543 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40666bf7-7907-4f54-8ba0-fa2369d4e805 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:34:57.142216 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40666bf7-7907-4f54-8ba0-fa2369d4e805 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:34:57.143816 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40666bf7-7907-4f54-8ba0-fa2369d4e805 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:34:57.154136 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1414/2813] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:34:57.159305 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f79e6ed-fde2-47f8-a207-d047b9d512ec admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:34:57.174051 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f79e6ed-fde2-47f8-a207-d047b9d512ec admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:34:57.179793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0b1c4-67f1-4688-94bf-4acd8e9dede0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:34:57.180457 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0b1c4-67f1-4688-94bf-4acd8e9dede0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:34:57.180730 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0b1c4-67f1-4688-94bf-4acd8e9dede0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:34:57.180995 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0b1c4-67f1-4688-94bf-4acd8e9dede0 admin admin] PATH_INFO: `/v3/projects/2f352b1f0e10410bbe9355b8ab354cd6/users/5123a3cd444f40c9864d644c29bc0fb1/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:34:57.194109 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:34:57.197352 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac0b1c4-67f1-4688-94bf-4acd8e9dede0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f352b1f0e10410bbe9355b8ab354cd6, user_id=5123a3cd444f40c9864d644c29bc0fb1, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:34:57.200231 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac0b1c4-67f1-4688-94bf-4acd8e9dede0 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:34:57.218293 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1400/2814] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:34:57 2022] PUT /identity/v3/projects/2f352b1f0e10410bbe9355b8ab354cd6/users/5123a3cd444f40c9864d644c29bc0fb1/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:34:57.222495 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40666bf7-7907-4f54-8ba0-fa2369d4e805 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:34:57.233101 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40666bf7-7907-4f54-8ba0-fa2369d4e805 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:34:57.271734 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:34:57.295307 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30af4d2b-43ca-44fe-976d-8e27c6330413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:34:57.295909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af4d2b-43ca-44fe-976d-8e27c6330413 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:34:57.296185 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af4d2b-43ca-44fe-976d-8e27c6330413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:34:57.296448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af4d2b-43ca-44fe-976d-8e27c6330413 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:34:57.297095 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30af4d2b-43ca-44fe-976d-8e27c6330413 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:34:57.298854 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30af4d2b-43ca-44fe-976d-8e27c6330413 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:34:57.308993 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1415/2815] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:34:57.313495 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0b1c4-67f1-4688-94bf-4acd8e9dede0 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:34:57.324473 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0b1c4-67f1-4688-94bf-4acd8e9dede0 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:34:57.329516 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd10af0a-aa97-4c09-9deb-b1b3d88aced5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:34:57.330188 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd10af0a-aa97-4c09-9deb-b1b3d88aced5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:34:57.330463 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd10af0a-aa97-4c09-9deb-b1b3d88aced5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:34:57.330728 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd10af0a-aa97-4c09-9deb-b1b3d88aced5 admin admin] PATH_INFO: `/v3/projects/2f352b1f0e10410bbe9355b8ab354cd6/users/5123a3cd444f40c9864d644c29bc0fb1/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:34:57.332664 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd10af0a-aa97-4c09-9deb-b1b3d88aced5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f352b1f0e10410bbe9355b8ab354cd6, user_id=5123a3cd444f40c9864d644c29bc0fb1, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:34:57.335328 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd10af0a-aa97-4c09-9deb-b1b3d88aced5 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:34:57.346571 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1401/2816] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:34:57 2022] PUT /identity/v3/projects/2f352b1f0e10410bbe9355b8ab354cd6/users/5123a3cd444f40c9864d644c29bc0fb1/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:34:57.357186 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fc786-2445-4b09-81c4-cd85b16be992 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:34:57.357697 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fc786-2445-4b09-81c4-cd85b16be992 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:34:57.358220 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fc786-2445-4b09-81c4-cd85b16be992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:34:57.391473 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-df1fc786-2445-4b09-81c4-cd85b16be992 None None] MFA Rules not processed for user `5123a3cd444f40c9864d644c29bc0fb1`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:34:57.392735 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-df1fc786-2445-4b09-81c4-cd85b16be992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:34:57.464648 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1416/2817] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:34:57 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:34:57.475807 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd10af0a-aa97-4c09-9deb-b1b3d88aced5 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:34:57.477824 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-dd10af0a-aa97-4c09-9deb-b1b3d88aced5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:34:57.540809 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd10af0a-aa97-4c09-9deb-b1b3d88aced5 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:34:57.545814 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68abf1f3-94eb-410d-9b7b-3e8572614072 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:34:57.546444 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68abf1f3-94eb-410d-9b7b-3e8572614072 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:34:57.546715 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68abf1f3-94eb-410d-9b7b-3e8572614072 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:34:57.546977 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68abf1f3-94eb-410d-9b7b-3e8572614072 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:34:57.557563 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68abf1f3-94eb-410d-9b7b-3e8572614072 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:34:57.559091 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68abf1f3-94eb-410d-9b7b-3e8572614072 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:34:57.568782 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1402/2818] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:34:57 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:34:58.088798 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1fc786-2445-4b09-81c4-cd85b16be992 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:34:58.090341 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-df1fc786-2445-4b09-81c4-cd85b16be992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:34:58.102757 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1fc786-2445-4b09-81c4-cd85b16be992 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:34:58.107102 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88523033-14cd-4d5d-a893-7a01d012bd07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:34:58.107701 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88523033-14cd-4d5d-a893-7a01d012bd07 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:34:58.107971 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88523033-14cd-4d5d-a893-7a01d012bd07 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:34:58.108238 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88523033-14cd-4d5d-a893-7a01d012bd07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:34:58.172005 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88523033-14cd-4d5d-a893-7a01d012bd07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:34:58.173568 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88523033-14cd-4d5d-a893-7a01d012bd07 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:34:58.185480 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1417/2819] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:34:58 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:35:02.244786 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68abf1f3-94eb-410d-9b7b-3e8572614072 service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:02.253994 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68abf1f3-94eb-410d-9b7b-3e8572614072 service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:02.258606 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6d9cd-4b2a-4c69-b95a-529cdb03b9da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:02.259194 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6d9cd-4b2a-4c69-b95a-529cdb03b9da service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:02.259470 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6d9cd-4b2a-4c69-b95a-529cdb03b9da service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:02.259707 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6d9cd-4b2a-4c69-b95a-529cdb03b9da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:02.315727 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac6d9cd-4b2a-4c69-b95a-529cdb03b9da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:02.317090 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac6d9cd-4b2a-4c69-b95a-529cdb03b9da service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:02.325820 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1403/2820] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:35:02 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:35:04.458592 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88523033-14cd-4d5d-a893-7a01d012bd07 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:04.520116 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:35:04.524661 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88523033-14cd-4d5d-a893-7a01d012bd07 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:04.529828 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762e9c17-aa0b-4b83-a285-c042a97f0026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:04.530493 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e9c17-aa0b-4b83-a285-c042a97f0026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:04.530768 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e9c17-aa0b-4b83-a285-c042a97f0026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:04.531029 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e9c17-aa0b-4b83-a285-c042a97f0026 admin admin] PATH_INFO: `/v3/users/8acb4c939dd747958ecc2676c9efeeaf` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:04.532208 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762e9c17-aa0b-4b83-a285-c042a97f0026 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8acb4c939dd747958ecc2676c9efeeaf)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:04.534176 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762e9c17-aa0b-4b83-a285-c042a97f0026 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:04.589995 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-762e9c17-aa0b-4b83-a285-c042a97f0026 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8acb4c939dd747958ecc2676c9efeeaf'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:04.596904 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-762e9c17-aa0b-4b83-a285-c042a97f0026 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8acb4c939dd747958ecc2676c9efeeaf'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:04.607387 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-762e9c17-aa0b-4b83-a285-c042a97f0026 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8acb4c939dd747958ecc2676c9efeeaf'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:04.608058 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-762e9c17-aa0b-4b83-a285-c042a97f0026 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8acb4c939dd747958ecc2676c9efeeaf'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:04.620834 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-762e9c17-aa0b-4b83-a285-c042a97f0026 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8acb4c939dd747958ecc2676c9efeeaf'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:04.626865 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1418/2821] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:35:04 2022] DELETE /identity/v3/users/8acb4c939dd747958ecc2676c9efeeaf => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:35:04.631458 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e0eef-37ba-4e3c-a672-78002fc8b020 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:04.632013 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e0eef-37ba-4e3c-a672-78002fc8b020 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:04.632013 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e0eef-37ba-4e3c-a672-78002fc8b020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:04.651998 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-379e0eef-37ba-4e3c-a672-78002fc8b020 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:35:04.653197 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-379e0eef-37ba-4e3c-a672-78002fc8b020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:35:04.719605 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1404/2822] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:35:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:35:04.730439 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762e9c17-aa0b-4b83-a285-c042a97f0026 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:04.732045 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-762e9c17-aa0b-4b83-a285-c042a97f0026 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:35:04.791514 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762e9c17-aa0b-4b83-a285-c042a97f0026 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:04.796089 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47bb594-5182-4f1e-9135-f6ae6542ac0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:04.796685 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47bb594-5182-4f1e-9135-f6ae6542ac0b service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:04.796958 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47bb594-5182-4f1e-9135-f6ae6542ac0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:04.797223 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47bb594-5182-4f1e-9135-f6ae6542ac0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:04.807024 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47bb594-5182-4f1e-9135-f6ae6542ac0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:04.808540 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47bb594-5182-4f1e-9135-f6ae6542ac0b service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:04.817834 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1419/2823] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:35:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:35:05.092533 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e0eef-37ba-4e3c-a672-78002fc8b020 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:05.094053 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-379e0eef-37ba-4e3c-a672-78002fc8b020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:35:05.108203 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e0eef-37ba-4e3c-a672-78002fc8b020 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:05.113839 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d27865a-05c4-48e4-8865-673335aae32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:05.114539 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27865a-05c4-48e4-8865-673335aae32d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:05.114813 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27865a-05c4-48e4-8865-673335aae32d admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:05.115072 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27865a-05c4-48e4-8865-673335aae32d admin admin] PATH_INFO: `/v3/projects/3eab51577af34120abf5eaa9a8ce8399` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:05.117099 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d27865a-05c4-48e4-8865-673335aae32d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3eab51577af34120abf5eaa9a8ce8399)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:05.118442 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d27865a-05c4-48e4-8865-673335aae32d admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:05.135912 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:35:05.297119 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-8d27865a-05c4-48e4-8865-673335aae32d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3eab51577af34120abf5eaa9a8ce8399', 'request_id': 'req-8d27865a-05c4-48e4-8865-673335aae32d'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:05.303906 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-8d27865a-05c4-48e4-8865-673335aae32d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3eab51577af34120abf5eaa9a8ce8399', 'request_id': 'req-8d27865a-05c4-48e4-8865-673335aae32d'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:05.309809 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-8d27865a-05c4-48e4-8865-673335aae32d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3eab51577af34120abf5eaa9a8ce8399', 'request_id': 'req-8d27865a-05c4-48e4-8865-673335aae32d'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:05.323142 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-8d27865a-05c4-48e4-8865-673335aae32d admin admin] The token cache is being invalidate because project 3eab51577af34120abf5eaa9a8ce8399 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:35:05.323452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-8d27865a-05c4-48e4-8865-673335aae32d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:05.324086 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-8d27865a-05c4-48e4-8865-673335aae32d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:05.325656 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1405/2824] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:35:05 2022] DELETE /identity/v3/projects/3eab51577af34120abf5eaa9a8ce8399 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:35:05.329976 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47bb594-5182-4f1e-9135-f6ae6542ac0b service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:05.341047 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47bb594-5182-4f1e-9135-f6ae6542ac0b service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:05.401681 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff6ea10-30e1-4ccc-b643-be40c22b09c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:05.402356 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6ea10-30e1-4ccc-b643-be40c22b09c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:05.402666 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6ea10-30e1-4ccc-b643-be40c22b09c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:05.402934 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6ea10-30e1-4ccc-b643-be40c22b09c7 admin admin] PATH_INFO: `/v3/users/2ec6494b595e4d6cae7a4f1fb558a970` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:05.404086 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff6ea10-30e1-4ccc-b643-be40c22b09c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ec6494b595e4d6cae7a4f1fb558a970)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:05.406066 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff6ea10-30e1-4ccc-b643-be40c22b09c7 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:05.427597 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:35:05.454987 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8ff6ea10-30e1-4ccc-b643-be40c22b09c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ec6494b595e4d6cae7a4f1fb558a970'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:05.461957 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8ff6ea10-30e1-4ccc-b643-be40c22b09c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ec6494b595e4d6cae7a4f1fb558a970'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:05.470818 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8ff6ea10-30e1-4ccc-b643-be40c22b09c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ec6494b595e4d6cae7a4f1fb558a970'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:05.471446 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8ff6ea10-30e1-4ccc-b643-be40c22b09c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ec6494b595e4d6cae7a4f1fb558a970'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:05.485656 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8ff6ea10-30e1-4ccc-b643-be40c22b09c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ec6494b595e4d6cae7a4f1fb558a970'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:05.491765 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1420/2825] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:35:05 2022] DELETE /identity/v3/users/2ec6494b595e4d6cae7a4f1fb558a970 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:35:05.768794 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d27865a-05c4-48e4-8865-673335aae32d admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:05.770293 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-8d27865a-05c4-48e4-8865-673335aae32d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:35:05.786035 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d27865a-05c4-48e4-8865-673335aae32d admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:05.843482 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1641a1-c987-4d1c-847d-db83ba9eef10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:05.844066 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1641a1-c987-4d1c-847d-db83ba9eef10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:05.844337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1641a1-c987-4d1c-847d-db83ba9eef10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:05.844604 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1641a1-c987-4d1c-847d-db83ba9eef10 admin admin] PATH_INFO: `/v3/projects/dcf1ad1aa60847cf9a74d4bd97e03bf5` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:05.845782 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1641a1-c987-4d1c-847d-db83ba9eef10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcf1ad1aa60847cf9a74d4bd97e03bf5)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:05.847732 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1641a1-c987-4d1c-847d-db83ba9eef10 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:05.898681 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-da1641a1-c987-4d1c-847d-db83ba9eef10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcf1ad1aa60847cf9a74d4bd97e03bf5', 'request_id': 'req-da1641a1-c987-4d1c-847d-db83ba9eef10'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:05.904960 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-da1641a1-c987-4d1c-847d-db83ba9eef10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcf1ad1aa60847cf9a74d4bd97e03bf5', 'request_id': 'req-da1641a1-c987-4d1c-847d-db83ba9eef10'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:05.910948 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-da1641a1-c987-4d1c-847d-db83ba9eef10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcf1ad1aa60847cf9a74d4bd97e03bf5', 'request_id': 'req-da1641a1-c987-4d1c-847d-db83ba9eef10'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:05.925921 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-da1641a1-c987-4d1c-847d-db83ba9eef10 admin admin] The token cache is being invalidate because project dcf1ad1aa60847cf9a74d4bd97e03bf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:35:05.926227 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-da1641a1-c987-4d1c-847d-db83ba9eef10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:05.926858 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-da1641a1-c987-4d1c-847d-db83ba9eef10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:05.928365 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1406/2826] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:35:05 2022] DELETE /identity/v3/projects/dcf1ad1aa60847cf9a74d4bd97e03bf5 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:35:05.932721 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff6ea10-30e1-4ccc-b643-be40c22b09c7 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:05.934254 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8ff6ea10-30e1-4ccc-b643-be40c22b09c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:35:05.948039 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff6ea10-30e1-4ccc-b643-be40c22b09c7 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:06.008548 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8ad6f6-e22f-4ae0-8f9c-d8c5ebf379a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:06.009175 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ad6f6-e22f-4ae0-8f9c-d8c5ebf379a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:06.009448 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ad6f6-e22f-4ae0-8f9c-d8c5ebf379a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:06.009709 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ad6f6-e22f-4ae0-8f9c-d8c5ebf379a5 admin admin] PATH_INFO: `/v3/users/5123a3cd444f40c9864d644c29bc0fb1` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:06.010908 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8ad6f6-e22f-4ae0-8f9c-d8c5ebf379a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5123a3cd444f40c9864d644c29bc0fb1)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:06.012834 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8ad6f6-e22f-4ae0-8f9c-d8c5ebf379a5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:06.063400 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0b8ad6f6-e22f-4ae0-8f9c-d8c5ebf379a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5123a3cd444f40c9864d644c29bc0fb1'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:06.070458 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0b8ad6f6-e22f-4ae0-8f9c-d8c5ebf379a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5123a3cd444f40c9864d644c29bc0fb1'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:06.078690 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0b8ad6f6-e22f-4ae0-8f9c-d8c5ebf379a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5123a3cd444f40c9864d644c29bc0fb1'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:06.079287 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0b8ad6f6-e22f-4ae0-8f9c-d8c5ebf379a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5123a3cd444f40c9864d644c29bc0fb1'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:06.091421 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0b8ad6f6-e22f-4ae0-8f9c-d8c5ebf379a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5123a3cd444f40c9864d644c29bc0fb1'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:06.097566 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1421/2827] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:35:05 2022] DELETE /identity/v3/users/5123a3cd444f40c9864d644c29bc0fb1 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:35:06.385515 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1641a1-c987-4d1c-847d-db83ba9eef10 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:06.387067 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-da1641a1-c987-4d1c-847d-db83ba9eef10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:35:06.399995 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1641a1-c987-4d1c-847d-db83ba9eef10 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:06.430300 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:35:06.460081 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d2e94-f84c-4053-88c0-373e88031ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:06.460712 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d2e94-f84c-4053-88c0-373e88031ccf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:06.461089 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d2e94-f84c-4053-88c0-373e88031ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:06.461216 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d2e94-f84c-4053-88c0-373e88031ccf admin admin] PATH_INFO: `/v3/projects/2f352b1f0e10410bbe9355b8ab354cd6` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:06.462400 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192d2e94-f84c-4053-88c0-373e88031ccf admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f352b1f0e10410bbe9355b8ab354cd6)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:06.464330 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192d2e94-f84c-4053-88c0-373e88031ccf admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:06.512811 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-192d2e94-f84c-4053-88c0-373e88031ccf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f352b1f0e10410bbe9355b8ab354cd6', 'request_id': 'req-192d2e94-f84c-4053-88c0-373e88031ccf'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:06.519038 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-192d2e94-f84c-4053-88c0-373e88031ccf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f352b1f0e10410bbe9355b8ab354cd6', 'request_id': 'req-192d2e94-f84c-4053-88c0-373e88031ccf'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:06.524907 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-192d2e94-f84c-4053-88c0-373e88031ccf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f352b1f0e10410bbe9355b8ab354cd6', 'request_id': 'req-192d2e94-f84c-4053-88c0-373e88031ccf'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:06.542590 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-192d2e94-f84c-4053-88c0-373e88031ccf admin admin] The token cache is being invalidate because project 2f352b1f0e10410bbe9355b8ab354cd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:35:06.542871 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-192d2e94-f84c-4053-88c0-373e88031ccf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:06.543613 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-192d2e94-f84c-4053-88c0-373e88031ccf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:06.545223 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1407/2828] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:35:06 2022] DELETE /identity/v3/projects/2f352b1f0e10410bbe9355b8ab354cd6 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:35:06.551539 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570558f-6018-4b97-a2ea-0afb6ace9bec None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:06.552033 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570558f-6018-4b97-a2ea-0afb6ace9bec None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:06.552285 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570558f-6018-4b97-a2ea-0afb6ace9bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:06.577012 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-c570558f-6018-4b97-a2ea-0afb6ace9bec None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:35:06.578252 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c570558f-6018-4b97-a2ea-0afb6ace9bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:35:06.591180 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:35:06.641814 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1422/2829] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:35:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:35:06.655743 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeef62f-6303-463f-82c3-9df99b54ed10 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:06.656035 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeef62f-6303-463f-82c3-9df99b54ed10 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:06.656407 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeef62f-6303-463f-82c3-9df99b54ed10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:06.679015 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-5eeef62f-6303-463f-82c3-9df99b54ed10 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:35:06.680210 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-5eeef62f-6303-463f-82c3-9df99b54ed10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:35:06.689603 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1408/2830] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:35:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:35:06.694065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c570558f-6018-4b97-a2ea-0afb6ace9bec None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:06.705127 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c570558f-6018-4b97-a2ea-0afb6ace9bec None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:06.711088 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d1414-7a6b-4d42-8f9c-d2d443f99dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:06.711692 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d1414-7a6b-4d42-8f9c-d2d443f99dcb admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:06.711969 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d1414-7a6b-4d42-8f9c-d2d443f99dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:06.712220 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d1414-7a6b-4d42-8f9c-d2d443f99dcb admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:06.712868 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d1414-7a6b-4d42-8f9c-d2d443f99dcb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:35:06.713167 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d1414-7a6b-4d42-8f9c-d2d443f99dcb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:06.714830 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d1414-7a6b-4d42-8f9c-d2d443f99dcb admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:06.728201 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1423/2831] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:35:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:35:06.732942 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeef62f-6303-463f-82c3-9df99b54ed10 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:06.742658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeef62f-6303-463f-82c3-9df99b54ed10 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:06.748755 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7589d555-3a2c-4ee9-ba98-4d00e21c0c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:06.749361 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7589d555-3a2c-4ee9-ba98-4d00e21c0c66 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:06.749637 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7589d555-3a2c-4ee9-ba98-4d00e21c0c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:06.749901 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7589d555-3a2c-4ee9-ba98-4d00e21c0c66 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:06.750570 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7589d555-3a2c-4ee9-ba98-4d00e21c0c66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:06.752508 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7589d555-3a2c-4ee9-ba98-4d00e21c0c66 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:06.782656 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1409/2832] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:35:06 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:35:06.787099 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d1414-7a6b-4d42-8f9c-d2d443f99dcb admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:06.797840 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d1414-7a6b-4d42-8f9c-d2d443f99dcb admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:06.871607 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c217da-17cd-42f6-8d28-1c08e32ff033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:06.872213 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c217da-17cd-42f6-8d28-1c08e32ff033 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:06.872485 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c217da-17cd-42f6-8d28-1c08e32ff033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:06.872751 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c217da-17cd-42f6-8d28-1c08e32ff033 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:06.873381 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c217da-17cd-42f6-8d28-1c08e32ff033 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:06.875334 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c217da-17cd-42f6-8d28-1c08e32ff033 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:06.899835 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1424/2833] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:35:06 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:35:06.904057 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7589d555-3a2c-4ee9-ba98-4d00e21c0c66 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:06.915009 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7589d555-3a2c-4ee9-ba98-4d00e21c0c66 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:06.919958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b828f5-ee2a-412b-9ae2-b66a46dfdb83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:06.920544 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b828f5-ee2a-412b-9ae2-b66a46dfdb83 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:06.920825 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b828f5-ee2a-412b-9ae2-b66a46dfdb83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:06.921085 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b828f5-ee2a-412b-9ae2-b66a46dfdb83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:06.921719 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b828f5-ee2a-412b-9ae2-b66a46dfdb83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:06.923355 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b828f5-ee2a-412b-9ae2-b66a46dfdb83 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:06.933564 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1410/2834] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:35:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:35:06.938299 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c217da-17cd-42f6-8d28-1c08e32ff033 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:06.952417 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c217da-17cd-42f6-8d28-1c08e32ff033 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:06.960144 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad133e4e-fe1e-4b3c-bd40-f6d40da81dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:06.960747 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad133e4e-fe1e-4b3c-bd40-f6d40da81dd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:06.961020 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad133e4e-fe1e-4b3c-bd40-f6d40da81dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:06.961282 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad133e4e-fe1e-4b3c-bd40-f6d40da81dd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:06.961937 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad133e4e-fe1e-4b3c-bd40-f6d40da81dd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:06.963578 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad133e4e-fe1e-4b3c-bd40-f6d40da81dd3 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:06.974928 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1425/2835] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:35:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:35:06.979595 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b828f5-ee2a-412b-9ae2-b66a46dfdb83 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:06.990581 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b828f5-ee2a-412b-9ae2-b66a46dfdb83 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:06.995912 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc1b9ac-3dea-402d-9774-dada8bd13b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:06.996561 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc1b9ac-3dea-402d-9774-dada8bd13b66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:06.996832 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc1b9ac-3dea-402d-9774-dada8bd13b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:06.997099 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc1b9ac-3dea-402d-9774-dada8bd13b66 admin admin] PATH_INFO: `/v3/projects/2200b5fbdadb4a17959983a6bd4cd4e4/users/1235cbe217294966a95626045e53b555/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:07.012092 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc1b9ac-3dea-402d-9774-dada8bd13b66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2200b5fbdadb4a17959983a6bd4cd4e4, user_id=1235cbe217294966a95626045e53b555, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:07.015003 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc1b9ac-3dea-402d-9774-dada8bd13b66 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:07.029315 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1411/2836] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:35:06 2022] PUT /identity/v3/projects/2200b5fbdadb4a17959983a6bd4cd4e4/users/1235cbe217294966a95626045e53b555/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:35:07.040557 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a8929-bda8-4b8e-839b-372b96230222 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:07.040845 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a8929-bda8-4b8e-839b-372b96230222 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:07.041122 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a8929-bda8-4b8e-839b-372b96230222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:07.069835 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-416a8929-bda8-4b8e-839b-372b96230222 None None] MFA Rules not processed for user `1235cbe217294966a95626045e53b555`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:35:07.071080 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-416a8929-bda8-4b8e-839b-372b96230222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:35:07.147164 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1426/2837] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:35:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:35:07.162229 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc1b9ac-3dea-402d-9774-dada8bd13b66 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:07.165026 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-2cc1b9ac-3dea-402d-9774-dada8bd13b66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:35:07.229610 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc1b9ac-3dea-402d-9774-dada8bd13b66 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:07.233622 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d64ca-2286-44ba-bcc3-5aee3ba5bb37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:07.234255 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d64ca-2286-44ba-bcc3-5aee3ba5bb37 service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:07.234537 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d64ca-2286-44ba-bcc3-5aee3ba5bb37 service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:07.234804 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d64ca-2286-44ba-bcc3-5aee3ba5bb37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:07.243784 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10d64ca-2286-44ba-bcc3-5aee3ba5bb37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:07.245214 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10d64ca-2286-44ba-bcc3-5aee3ba5bb37 service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:07.254483 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1412/2838] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:35:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:35:15.774220 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a8929-bda8-4b8e-839b-372b96230222 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:15.789732 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:35:15.840616 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a8929-bda8-4b8e-839b-372b96230222 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:15.846966 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff50854-1c0e-4bce-b8c9-ba2f853e8897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:15.847571 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff50854-1c0e-4bce-b8c9-ba2f853e8897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:15.847854 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff50854-1c0e-4bce-b8c9-ba2f853e8897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:15.848150 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff50854-1c0e-4bce-b8c9-ba2f853e8897 admin admin] PATH_INFO: `/v3/users/1235cbe217294966a95626045e53b555` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:15.849468 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff50854-1c0e-4bce-b8c9-ba2f853e8897 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1235cbe217294966a95626045e53b555)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:15.851453 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff50854-1c0e-4bce-b8c9-ba2f853e8897 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:15.909995 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7ff50854-1c0e-4bce-b8c9-ba2f853e8897 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1235cbe217294966a95626045e53b555'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:15.917939 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7ff50854-1c0e-4bce-b8c9-ba2f853e8897 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1235cbe217294966a95626045e53b555'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:15.927825 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7ff50854-1c0e-4bce-b8c9-ba2f853e8897 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1235cbe217294966a95626045e53b555'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:15.928705 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7ff50854-1c0e-4bce-b8c9-ba2f853e8897 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1235cbe217294966a95626045e53b555'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:15.946527 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7ff50854-1c0e-4bce-b8c9-ba2f853e8897 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1235cbe217294966a95626045e53b555'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:15.952570 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1427/2839] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:35:15 2022] DELETE /identity/v3/users/1235cbe217294966a95626045e53b555 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:35:15.957186 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e72c6-c05a-41d5-9933-ad93405a0466 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:15.957718 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e72c6-c05a-41d5-9933-ad93405a0466 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:15.957718 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e72c6-c05a-41d5-9933-ad93405a0466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:15.965144 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:35:15.980086 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-a59e72c6-c05a-41d5-9933-ad93405a0466 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:35:15.981283 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a59e72c6-c05a-41d5-9933-ad93405a0466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:35:16.041187 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1413/2840] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:35:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:35:16.051318 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff50854-1c0e-4bce-b8c9-ba2f853e8897 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:16.052887 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7ff50854-1c0e-4bce-b8c9-ba2f853e8897 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:35:16.112290 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff50854-1c0e-4bce-b8c9-ba2f853e8897 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:16.117703 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889172ec-881e-4a68-93d7-8634ac8f652e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:16.118630 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889172ec-881e-4a68-93d7-8634ac8f652e service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:16.118941 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889172ec-881e-4a68-93d7-8634ac8f652e service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:16.119231 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889172ec-881e-4a68-93d7-8634ac8f652e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:16.129302 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889172ec-881e-4a68-93d7-8634ac8f652e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:16.130804 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889172ec-881e-4a68-93d7-8634ac8f652e service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:16.140521 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1428/2841] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:35:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:35:16.426758 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59e72c6-c05a-41d5-9933-ad93405a0466 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:16.428300 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a59e72c6-c05a-41d5-9933-ad93405a0466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:35:16.441175 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59e72c6-c05a-41d5-9933-ad93405a0466 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:16.446264 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9759b-8cb3-43cd-b321-f998c3cb683c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:16.447425 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9759b-8cb3-43cd-b321-f998c3cb683c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:16.447714 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9759b-8cb3-43cd-b321-f998c3cb683c admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:16.448011 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9759b-8cb3-43cd-b321-f998c3cb683c admin admin] PATH_INFO: `/v3/projects/2200b5fbdadb4a17959983a6bd4cd4e4` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:16.449453 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d9759b-8cb3-43cd-b321-f998c3cb683c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2200b5fbdadb4a17959983a6bd4cd4e4)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:16.451224 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d9759b-8cb3-43cd-b321-f998c3cb683c admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:16.506658 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-b4d9759b-8cb3-43cd-b321-f998c3cb683c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2200b5fbdadb4a17959983a6bd4cd4e4', 'request_id': 'req-b4d9759b-8cb3-43cd-b321-f998c3cb683c'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:16.513608 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-b4d9759b-8cb3-43cd-b321-f998c3cb683c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2200b5fbdadb4a17959983a6bd4cd4e4', 'request_id': 'req-b4d9759b-8cb3-43cd-b321-f998c3cb683c'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:16.519107 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-b4d9759b-8cb3-43cd-b321-f998c3cb683c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2200b5fbdadb4a17959983a6bd4cd4e4', 'request_id': 'req-b4d9759b-8cb3-43cd-b321-f998c3cb683c'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:16.532693 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-b4d9759b-8cb3-43cd-b321-f998c3cb683c admin admin] The token cache is being invalidate because project 2200b5fbdadb4a17959983a6bd4cd4e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:35:16.532974 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-b4d9759b-8cb3-43cd-b321-f998c3cb683c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:16.533625 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-b4d9759b-8cb3-43cd-b321-f998c3cb683c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:16.535194 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1414/2842] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:35:16 2022] DELETE /identity/v3/projects/2200b5fbdadb4a17959983a6bd4cd4e4 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:35:16.540349 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca5c7c-c127-4a3e-8dbc-4aec0fa4ddce None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:16.540636 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca5c7c-c127-4a3e-8dbc-4aec0fa4ddce None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:16.540974 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca5c7c-c127-4a3e-8dbc-4aec0fa4ddce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:16.560277 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-4fca5c7c-c127-4a3e-8dbc-4aec0fa4ddce None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:35:16.561523 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4fca5c7c-c127-4a3e-8dbc-4aec0fa4ddce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:35:16.593659 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:35:16.625198 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1429/2843] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:35:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:35:16.633175 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc973ac-e259-42d7-a0e2-f783bf180775 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:16.633471 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc973ac-e259-42d7-a0e2-f783bf180775 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:16.633733 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc973ac-e259-42d7-a0e2-f783bf180775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:16.654226 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-edc973ac-e259-42d7-a0e2-f783bf180775 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:35:16.655404 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-edc973ac-e259-42d7-a0e2-f783bf180775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:35:16.663561 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1415/2844] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:35:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:35:16.668095 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca5c7c-c127-4a3e-8dbc-4aec0fa4ddce None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:16.678500 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca5c7c-c127-4a3e-8dbc-4aec0fa4ddce None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:16.684129 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0912e-92ee-4bd0-b613-f4cdee1c2f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:16.684721 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0912e-92ee-4bd0-b613-f4cdee1c2f0d admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:16.684992 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0912e-92ee-4bd0-b613-f4cdee1c2f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:16.685250 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0912e-92ee-4bd0-b613-f4cdee1c2f0d admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:16.685888 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c0912e-92ee-4bd0-b613-f4cdee1c2f0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:35:16.686225 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c0912e-92ee-4bd0-b613-f4cdee1c2f0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:16.687803 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c0912e-92ee-4bd0-b613-f4cdee1c2f0d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:16.699935 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1430/2845] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:35:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:35:16.704694 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc973ac-e259-42d7-a0e2-f783bf180775 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:16.718993 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc973ac-e259-42d7-a0e2-f783bf180775 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:16.727674 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb043a-2fe3-414d-b569-1d4408521a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:16.728554 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb043a-2fe3-414d-b569-1d4408521a06 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:16.729171 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb043a-2fe3-414d-b569-1d4408521a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:16.729436 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb043a-2fe3-414d-b569-1d4408521a06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:16.730051 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fb043a-2fe3-414d-b569-1d4408521a06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:16.731979 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fb043a-2fe3-414d-b569-1d4408521a06 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:16.761585 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1416/2846] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:35:16 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:35:16.766027 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0912e-92ee-4bd0-b613-f4cdee1c2f0d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:16.776152 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0912e-92ee-4bd0-b613-f4cdee1c2f0d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:16.835553 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc3159-5d7e-471f-b189-ba9b3a6f600e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:16.836152 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc3159-5d7e-471f-b189-ba9b3a6f600e admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:16.836556 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc3159-5d7e-471f-b189-ba9b3a6f600e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:16.836826 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc3159-5d7e-471f-b189-ba9b3a6f600e admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:16.837491 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddc3159-5d7e-471f-b189-ba9b3a6f600e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:16.839457 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddc3159-5d7e-471f-b189-ba9b3a6f600e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:16.865671 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1431/2847] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:35:16 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:35:16.868952 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb043a-2fe3-414d-b569-1d4408521a06 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:16.879833 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb043a-2fe3-414d-b569-1d4408521a06 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:16.885302 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19f5769-cb83-439f-be23-57dc4e061734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:16.885897 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f5769-cb83-439f-be23-57dc4e061734 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:16.886203 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f5769-cb83-439f-be23-57dc4e061734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:16.886466 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f5769-cb83-439f-be23-57dc4e061734 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:16.887110 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19f5769-cb83-439f-be23-57dc4e061734 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:16.888726 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19f5769-cb83-439f-be23-57dc4e061734 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:16.899249 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1417/2848] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:35:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:35:16.903695 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc3159-5d7e-471f-b189-ba9b3a6f600e admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:16.914375 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc3159-5d7e-471f-b189-ba9b3a6f600e admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:16.920289 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c030ab12-e7b9-49a0-8674-c9870b89b6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:16.920911 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030ab12-e7b9-49a0-8674-c9870b89b6d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:16.921186 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030ab12-e7b9-49a0-8674-c9870b89b6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:16.921449 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030ab12-e7b9-49a0-8674-c9870b89b6d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:16.922124 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c030ab12-e7b9-49a0-8674-c9870b89b6d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:16.923735 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c030ab12-e7b9-49a0-8674-c9870b89b6d5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:16.935101 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1432/2849] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:35:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:35:16.939288 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19f5769-cb83-439f-be23-57dc4e061734 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:16.949457 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19f5769-cb83-439f-be23-57dc4e061734 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:16.954303 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ba9c26-b055-4518-9a88-c230460eaa5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:16.954945 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba9c26-b055-4518-9a88-c230460eaa5a admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:16.955212 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba9c26-b055-4518-9a88-c230460eaa5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:16.955518 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba9c26-b055-4518-9a88-c230460eaa5a admin admin] PATH_INFO: `/v3/projects/3fed5d110f15431bb19e78a8e1bfbe55/users/438f90e913bc480bbfb706bad0db5d94/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:16.970581 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ba9c26-b055-4518-9a88-c230460eaa5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fed5d110f15431bb19e78a8e1bfbe55, user_id=438f90e913bc480bbfb706bad0db5d94, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:16.973277 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ba9c26-b055-4518-9a88-c230460eaa5a admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:16.988473 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1418/2850] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:35:16 2022] PUT /identity/v3/projects/3fed5d110f15431bb19e78a8e1bfbe55/users/438f90e913bc480bbfb706bad0db5d94/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:35:16.998877 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7d464-fffb-4a71-97cd-848ef97c5309 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:16.999189 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7d464-fffb-4a71-97cd-848ef97c5309 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:16.999483 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7d464-fffb-4a71-97cd-848ef97c5309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:17.028048 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-23b7d464-fffb-4a71-97cd-848ef97c5309 None None] MFA Rules not processed for user `438f90e913bc480bbfb706bad0db5d94`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:35:17.029183 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-23b7d464-fffb-4a71-97cd-848ef97c5309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:35:17.094597 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1433/2851] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:35:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:35:17.099292 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ba9c26-b055-4518-9a88-c230460eaa5a admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:17.125372 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:35:17.164065 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ba9c26-b055-4518-9a88-c230460eaa5a admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:17.169601 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e205838-bfc1-4656-bed7-6aa17bef1227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:17.170247 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e205838-bfc1-4656-bed7-6aa17bef1227 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:17.170526 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e205838-bfc1-4656-bed7-6aa17bef1227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:17.170793 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e205838-bfc1-4656-bed7-6aa17bef1227 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:17.171472 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e205838-bfc1-4656-bed7-6aa17bef1227 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:17.173390 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e205838-bfc1-4656-bed7-6aa17bef1227 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:17.202661 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1419/2852] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:35:17 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:35:17.206930 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b7d464-fffb-4a71-97cd-848ef97c5309 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:17.218120 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b7d464-fffb-4a71-97cd-848ef97c5309 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:17.247805 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:35:17.276826 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9605bbe-1dd8-4efa-a51c-6366878fa8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:17.277428 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9605bbe-1dd8-4efa-a51c-6366878fa8dd admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:17.277701 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9605bbe-1dd8-4efa-a51c-6366878fa8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:17.277967 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9605bbe-1dd8-4efa-a51c-6366878fa8dd admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:17.278678 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9605bbe-1dd8-4efa-a51c-6366878fa8dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:17.280540 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9605bbe-1dd8-4efa-a51c-6366878fa8dd admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:17.304149 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1434/2853] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:35:17 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:35:17.310760 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e205838-bfc1-4656-bed7-6aa17bef1227 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:17.325439 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e205838-bfc1-4656-bed7-6aa17bef1227 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:17.330366 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8e24a-3f1a-4d96-b89b-081a2baa7482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:17.330954 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8e24a-3f1a-4d96-b89b-081a2baa7482 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:17.331228 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8e24a-3f1a-4d96-b89b-081a2baa7482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:17.331515 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8e24a-3f1a-4d96-b89b-081a2baa7482 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:17.332149 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c8e24a-3f1a-4d96-b89b-081a2baa7482 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:17.333715 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c8e24a-3f1a-4d96-b89b-081a2baa7482 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:17.344301 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1420/2854] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:35:17.348232 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9605bbe-1dd8-4efa-a51c-6366878fa8dd admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:17.358769 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9605bbe-1dd8-4efa-a51c-6366878fa8dd admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:17.363518 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb871677-ed6d-4fe2-af7e-94b3ebf54dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:17.364111 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb871677-ed6d-4fe2-af7e-94b3ebf54dfb admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:17.364385 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb871677-ed6d-4fe2-af7e-94b3ebf54dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:17.364649 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb871677-ed6d-4fe2-af7e-94b3ebf54dfb admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:17.365277 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb871677-ed6d-4fe2-af7e-94b3ebf54dfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:17.366886 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb871677-ed6d-4fe2-af7e-94b3ebf54dfb admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:17.376703 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1435/2855] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:35:17.380751 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8e24a-3f1a-4d96-b89b-081a2baa7482 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:17.390884 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8e24a-3f1a-4d96-b89b-081a2baa7482 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:17.395614 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3691ae91-de62-42b0-a394-ecd7bdebfed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:17.396247 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691ae91-de62-42b0-a394-ecd7bdebfed1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:17.396521 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691ae91-de62-42b0-a394-ecd7bdebfed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:17.396790 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691ae91-de62-42b0-a394-ecd7bdebfed1 admin admin] PATH_INFO: `/v3/projects/889109189941442eb1931e7f2466120a/users/4acc033f4c4c456a91c27d1643442cc2/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:17.411010 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3691ae91-de62-42b0-a394-ecd7bdebfed1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=889109189941442eb1931e7f2466120a, user_id=4acc033f4c4c456a91c27d1643442cc2, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:17.413636 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3691ae91-de62-42b0-a394-ecd7bdebfed1 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:17.432236 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1421/2856] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:35:17 2022] PUT /identity/v3/projects/889109189941442eb1931e7f2466120a/users/4acc033f4c4c456a91c27d1643442cc2/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:35:17.447502 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce65fa9-a083-49a9-a7df-dce49c4364f3 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:17.447784 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce65fa9-a083-49a9-a7df-dce49c4364f3 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:17.448056 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce65fa9-a083-49a9-a7df-dce49c4364f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:17.476881 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-7ce65fa9-a083-49a9-a7df-dce49c4364f3 None None] MFA Rules not processed for user `4acc033f4c4c456a91c27d1643442cc2`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:35:17.478119 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7ce65fa9-a083-49a9-a7df-dce49c4364f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:35:17.553210 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1436/2857] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:35:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:35:17.559870 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3691ae91-de62-42b0-a394-ecd7bdebfed1 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:17.627630 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3691ae91-de62-42b0-a394-ecd7bdebfed1 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:17.633266 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ceeb5-69e3-4f68-9184-4b30f3ae038d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:17.633869 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ceeb5-69e3-4f68-9184-4b30f3ae038d admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:17.634173 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ceeb5-69e3-4f68-9184-4b30f3ae038d admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:17.634439 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ceeb5-69e3-4f68-9184-4b30f3ae038d admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:17.635079 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9ceeb5-69e3-4f68-9184-4b30f3ae038d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:17.637022 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9ceeb5-69e3-4f68-9184-4b30f3ae038d admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:17.666471 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1422/2858] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:35:17 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:35:17.670878 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce65fa9-a083-49a9-a7df-dce49c4364f3 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:17.681821 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce65fa9-a083-49a9-a7df-dce49c4364f3 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:17.739700 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd93f053-854d-4f72-8b87-b3a6e08cb09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:17.740310 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd93f053-854d-4f72-8b87-b3a6e08cb09f admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:17.740608 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd93f053-854d-4f72-8b87-b3a6e08cb09f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:17.740880 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd93f053-854d-4f72-8b87-b3a6e08cb09f admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:17.741541 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd93f053-854d-4f72-8b87-b3a6e08cb09f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:17.743597 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd93f053-854d-4f72-8b87-b3a6e08cb09f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:17.766738 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1437/2859] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:35:17 2022] POST /identity/v3/users => generated 298 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:35:17.770929 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ceeb5-69e3-4f68-9184-4b30f3ae038d admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:17.781734 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ceeb5-69e3-4f68-9184-4b30f3ae038d admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:17.787617 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd859d-d123-498b-b212-5fbad71e0a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:17.788218 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd859d-d123-498b-b212-5fbad71e0a05 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:17.788499 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd859d-d123-498b-b212-5fbad71e0a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:17.788768 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd859d-d123-498b-b212-5fbad71e0a05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:17.789416 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd859d-d123-498b-b212-5fbad71e0a05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:17.791072 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd859d-d123-498b-b212-5fbad71e0a05 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:17.802281 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1423/2860] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:35:17.806700 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd93f053-854d-4f72-8b87-b3a6e08cb09f admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:17.816906 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd93f053-854d-4f72-8b87-b3a6e08cb09f admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:17.821786 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4fcd8d-e9c2-4fc3-9153-61ac76d97f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:17.822450 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fcd8d-e9c2-4fc3-9153-61ac76d97f58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:17.822724 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fcd8d-e9c2-4fc3-9153-61ac76d97f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:17.822989 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fcd8d-e9c2-4fc3-9153-61ac76d97f58 admin admin] PATH_INFO: `/v3/projects/1c2e8b25b468405a93b70e34294d978b/users/c686de5175df424382c6de5ffe1dc256/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:17.837795 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4fcd8d-e9c2-4fc3-9153-61ac76d97f58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c2e8b25b468405a93b70e34294d978b, user_id=c686de5175df424382c6de5ffe1dc256, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:17.840461 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4fcd8d-e9c2-4fc3-9153-61ac76d97f58 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:17.854726 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1438/2861] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:35:17 2022] PUT /identity/v3/projects/1c2e8b25b468405a93b70e34294d978b/users/c686de5175df424382c6de5ffe1dc256/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:35:17.858785 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd859d-d123-498b-b212-5fbad71e0a05 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:17.868729 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd859d-d123-498b-b212-5fbad71e0a05 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:17.911454 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:35:17.926816 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b15fe4a-5e2d-462b-99c8-2b2bfb7e3c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:17.927432 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15fe4a-5e2d-462b-99c8-2b2bfb7e3c8a admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:17.927727 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15fe4a-5e2d-462b-99c8-2b2bfb7e3c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:17.927995 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15fe4a-5e2d-462b-99c8-2b2bfb7e3c8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:17.928641 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b15fe4a-5e2d-462b-99c8-2b2bfb7e3c8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:17.930335 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b15fe4a-5e2d-462b-99c8-2b2bfb7e3c8a admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:17.940202 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1424/2862] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:35:17.944387 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4fcd8d-e9c2-4fc3-9153-61ac76d97f58 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:17.951124 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:35:17.955115 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4fcd8d-e9c2-4fc3-9153-61ac76d97f58 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:17.960937 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84891ac-d7ce-4a74-84f3-eb75fae0e5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:17.961582 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84891ac-d7ce-4a74-84f3-eb75fae0e5f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:17.961857 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84891ac-d7ce-4a74-84f3-eb75fae0e5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:17.962336 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84891ac-d7ce-4a74-84f3-eb75fae0e5f2 admin admin] PATH_INFO: `/v3/projects/1c2e8b25b468405a93b70e34294d978b/users/c686de5175df424382c6de5ffe1dc256/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:17.964369 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84891ac-d7ce-4a74-84f3-eb75fae0e5f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c2e8b25b468405a93b70e34294d978b, user_id=c686de5175df424382c6de5ffe1dc256, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:17.967065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84891ac-d7ce-4a74-84f3-eb75fae0e5f2 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:17.977859 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1439/2863] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:35:17 2022] PUT /identity/v3/projects/1c2e8b25b468405a93b70e34294d978b/users/c686de5175df424382c6de5ffe1dc256/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:35:17.989545 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef4b2b-77bf-48df-9b1a-0fa8841db1d6 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:17.990077 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef4b2b-77bf-48df-9b1a-0fa8841db1d6 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:17.990203 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef4b2b-77bf-48df-9b1a-0fa8841db1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:18.020374 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-22ef4b2b-77bf-48df-9b1a-0fa8841db1d6 None None] MFA Rules not processed for user `c686de5175df424382c6de5ffe1dc256`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:35:18.022277 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-22ef4b2b-77bf-48df-9b1a-0fa8841db1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:35:18.100899 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1425/2864] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:35:17 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:35:18.113044 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84891ac-d7ce-4a74-84f3-eb75fae0e5f2 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:18.114801 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a84891ac-d7ce-4a74-84f3-eb75fae0e5f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:35:18.174157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84891ac-d7ce-4a74-84f3-eb75fae0e5f2 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:18.178318 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b9205f-2437-44b2-9b92-9b974388fece service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:18.178917 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9205f-2437-44b2-9b92-9b974388fece service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:18.179157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9205f-2437-44b2-9b92-9b974388fece service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:18.179431 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9205f-2437-44b2-9b92-9b974388fece service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:18.235836 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b9205f-2437-44b2-9b92-9b974388fece service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:18.237263 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b9205f-2437-44b2-9b92-9b974388fece service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:18.246261 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1440/2865] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:35:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:35:19.788436 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef4b2b-77bf-48df-9b1a-0fa8841db1d6 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:19.789995 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-22ef4b2b-77bf-48df-9b1a-0fa8841db1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:35:19.803548 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef4b2b-77bf-48df-9b1a-0fa8841db1d6 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:19.807948 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d92e65a-57e6-4070-b564-8d1d88e870ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:19.808553 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92e65a-57e6-4070-b564-8d1d88e870ef service cinder] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:19.808832 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92e65a-57e6-4070-b564-8d1d88e870ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:19.809103 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92e65a-57e6-4070-b564-8d1d88e870ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:19.870816 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d92e65a-57e6-4070-b564-8d1d88e870ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:19.872176 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d92e65a-57e6-4070-b564-8d1d88e870ef service cinder] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:19.881233 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1426/2866] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:35:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:35:20.439639 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b9205f-2437-44b2-9b92-9b974388fece service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:20.448868 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b9205f-2437-44b2-9b92-9b974388fece service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:20.453571 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1284c40-cd8c-48bd-8e0d-918e0a888277 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:20.454212 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1284c40-cd8c-48bd-8e0d-918e0a888277 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:20.454488 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1284c40-cd8c-48bd-8e0d-918e0a888277 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:20.454754 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1284c40-cd8c-48bd-8e0d-918e0a888277 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:20.464723 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1284c40-cd8c-48bd-8e0d-918e0a888277 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:20.466202 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1284c40-cd8c-48bd-8e0d-918e0a888277 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:20.475893 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1441/2867] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:35:20 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:35:24.414603 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d92e65a-57e6-4070-b564-8d1d88e870ef service cinder] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:24.485874 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d92e65a-57e6-4070-b564-8d1d88e870ef service cinder] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:24.491058 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afb41f-6599-4b56-bc79-7631a80711b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:24.491683 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afb41f-6599-4b56-bc79-7631a80711b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:24.491960 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afb41f-6599-4b56-bc79-7631a80711b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:24.492227 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afb41f-6599-4b56-bc79-7631a80711b3 admin admin] PATH_INFO: `/v3/users/438f90e913bc480bbfb706bad0db5d94` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:24.493483 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88afb41f-6599-4b56-bc79-7631a80711b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=438f90e913bc480bbfb706bad0db5d94)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:24.495577 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88afb41f-6599-4b56-bc79-7631a80711b3 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:24.501346 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:35:24.545678 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-88afb41f-6599-4b56-bc79-7631a80711b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '438f90e913bc480bbfb706bad0db5d94'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:24.552468 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-88afb41f-6599-4b56-bc79-7631a80711b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '438f90e913bc480bbfb706bad0db5d94'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:24.553114 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-88afb41f-6599-4b56-bc79-7631a80711b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '438f90e913bc480bbfb706bad0db5d94'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:24.562491 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-88afb41f-6599-4b56-bc79-7631a80711b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '438f90e913bc480bbfb706bad0db5d94'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:24.563331 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-88afb41f-6599-4b56-bc79-7631a80711b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '438f90e913bc480bbfb706bad0db5d94'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:24.580038 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1427/2868] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:35:24 2022] DELETE /identity/v3/users/438f90e913bc480bbfb706bad0db5d94 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:35:24.584945 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8192c7f-b977-4411-b38e-c6f1c99784e0 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:24.585434 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8192c7f-b977-4411-b38e-c6f1c99784e0 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:24.585545 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8192c7f-b977-4411-b38e-c6f1c99784e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:24.606036 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e8192c7f-b977-4411-b38e-c6f1c99784e0 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:35:24.607242 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e8192c7f-b977-4411-b38e-c6f1c99784e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:35:24.670794 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1442/2869] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:35:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:35:24.681118 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afb41f-6599-4b56-bc79-7631a80711b3 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:24.682766 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-88afb41f-6599-4b56-bc79-7631a80711b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:35:24.742075 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afb41f-6599-4b56-bc79-7631a80711b3 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:24.746168 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f98c02-1652-4cf2-be6a-a7860ef35a12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:24.746762 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f98c02-1652-4cf2-be6a-a7860ef35a12 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:24.747041 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f98c02-1652-4cf2-be6a-a7860ef35a12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:24.747305 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f98c02-1652-4cf2-be6a-a7860ef35a12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:24.756487 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f98c02-1652-4cf2-be6a-a7860ef35a12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:24.757940 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f98c02-1652-4cf2-be6a-a7860ef35a12 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:24.767588 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1428/2870] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:35:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:35:25.046980 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8192c7f-b977-4411-b38e-c6f1c99784e0 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:25.048521 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e8192c7f-b977-4411-b38e-c6f1c99784e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:35:25.062670 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8192c7f-b977-4411-b38e-c6f1c99784e0 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:25.067552 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed6203e-e69b-4268-a0af-a04e006b6de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:25.068211 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6203e-e69b-4268-a0af-a04e006b6de1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:25.068475 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6203e-e69b-4268-a0af-a04e006b6de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:25.068725 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6203e-e69b-4268-a0af-a04e006b6de1 admin admin] PATH_INFO: `/v3/projects/3fed5d110f15431bb19e78a8e1bfbe55` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:25.069886 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed6203e-e69b-4268-a0af-a04e006b6de1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fed5d110f15431bb19e78a8e1bfbe55)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:25.071896 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed6203e-e69b-4268-a0af-a04e006b6de1 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:25.105004 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:35:25.120831 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4ed6203e-e69b-4268-a0af-a04e006b6de1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fed5d110f15431bb19e78a8e1bfbe55', 'request_id': 'req-4ed6203e-e69b-4268-a0af-a04e006b6de1'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:25.126869 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4ed6203e-e69b-4268-a0af-a04e006b6de1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fed5d110f15431bb19e78a8e1bfbe55', 'request_id': 'req-4ed6203e-e69b-4268-a0af-a04e006b6de1'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:25.136123 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4ed6203e-e69b-4268-a0af-a04e006b6de1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fed5d110f15431bb19e78a8e1bfbe55', 'request_id': 'req-4ed6203e-e69b-4268-a0af-a04e006b6de1'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:25.145970 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4ed6203e-e69b-4268-a0af-a04e006b6de1 admin admin] The token cache is being invalidate because project 3fed5d110f15431bb19e78a8e1bfbe55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:35:25.146275 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4ed6203e-e69b-4268-a0af-a04e006b6de1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:25.146907 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4ed6203e-e69b-4268-a0af-a04e006b6de1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:25.148384 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1443/2871] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:35:25 2022] DELETE /identity/v3/projects/3fed5d110f15431bb19e78a8e1bfbe55 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:35:25.152622 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f98c02-1652-4cf2-be6a-a7860ef35a12 service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:25.162278 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f98c02-1652-4cf2-be6a-a7860ef35a12 service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:25.220643 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3465e07-72a4-4f71-8536-2cc3be911198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:25.221271 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3465e07-72a4-4f71-8536-2cc3be911198 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:25.221544 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3465e07-72a4-4f71-8536-2cc3be911198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:25.221809 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3465e07-72a4-4f71-8536-2cc3be911198 admin admin] PATH_INFO: `/v3/users/4acc033f4c4c456a91c27d1643442cc2` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:25.222976 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3465e07-72a4-4f71-8536-2cc3be911198 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4acc033f4c4c456a91c27d1643442cc2)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:25.224897 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3465e07-72a4-4f71-8536-2cc3be911198 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:25.261410 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:35:25.275932 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-e3465e07-72a4-4f71-8536-2cc3be911198 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4acc033f4c4c456a91c27d1643442cc2'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:25.282953 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-e3465e07-72a4-4f71-8536-2cc3be911198 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4acc033f4c4c456a91c27d1643442cc2'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:25.283657 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-e3465e07-72a4-4f71-8536-2cc3be911198 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4acc033f4c4c456a91c27d1643442cc2'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:25.291859 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-e3465e07-72a4-4f71-8536-2cc3be911198 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4acc033f4c4c456a91c27d1643442cc2'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:25.292513 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-e3465e07-72a4-4f71-8536-2cc3be911198 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4acc033f4c4c456a91c27d1643442cc2'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:25.309895 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1429/2872] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:35:25 2022] DELETE /identity/v3/users/4acc033f4c4c456a91c27d1643442cc2 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:35:25.598237 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed6203e-e69b-4268-a0af-a04e006b6de1 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:25.599706 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4ed6203e-e69b-4268-a0af-a04e006b6de1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:35:25.613668 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed6203e-e69b-4268-a0af-a04e006b6de1 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:25.673210 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd85b78-a010-468c-95ea-634b16394dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:25.673848 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd85b78-a010-468c-95ea-634b16394dec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:25.674192 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd85b78-a010-468c-95ea-634b16394dec admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:25.674460 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd85b78-a010-468c-95ea-634b16394dec admin admin] PATH_INFO: `/v3/projects/889109189941442eb1931e7f2466120a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:25.675612 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd85b78-a010-468c-95ea-634b16394dec admin admin] RBAC: Authorizing `identity:delete_project(project_id=889109189941442eb1931e7f2466120a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:25.677532 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd85b78-a010-468c-95ea-634b16394dec admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:25.726618 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dfd85b78-a010-468c-95ea-634b16394dec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '889109189941442eb1931e7f2466120a', 'request_id': 'req-dfd85b78-a010-468c-95ea-634b16394dec'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:25.733270 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dfd85b78-a010-468c-95ea-634b16394dec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '889109189941442eb1931e7f2466120a', 'request_id': 'req-dfd85b78-a010-468c-95ea-634b16394dec'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:25.742315 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dfd85b78-a010-468c-95ea-634b16394dec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '889109189941442eb1931e7f2466120a', 'request_id': 'req-dfd85b78-a010-468c-95ea-634b16394dec'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:25.752398 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dfd85b78-a010-468c-95ea-634b16394dec admin admin] The token cache is being invalidate because project 889109189941442eb1931e7f2466120a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:35:25.752682 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dfd85b78-a010-468c-95ea-634b16394dec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:25.753300 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dfd85b78-a010-468c-95ea-634b16394dec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:25.754806 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1444/2873] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:35:25 2022] DELETE /identity/v3/projects/889109189941442eb1931e7f2466120a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:35:25.758857 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3465e07-72a4-4f71-8536-2cc3be911198 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:25.760465 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-e3465e07-72a4-4f71-8536-2cc3be911198 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:35:25.779540 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3465e07-72a4-4f71-8536-2cc3be911198 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:25.856169 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b7f24-ee79-4c5a-8fc3-ddb7c1d4caee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:25.856720 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b7f24-ee79-4c5a-8fc3-ddb7c1d4caee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:25.856997 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b7f24-ee79-4c5a-8fc3-ddb7c1d4caee admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:25.857267 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b7f24-ee79-4c5a-8fc3-ddb7c1d4caee admin admin] PATH_INFO: `/v3/users/c686de5175df424382c6de5ffe1dc256` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:25.858566 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b7f24-ee79-4c5a-8fc3-ddb7c1d4caee admin admin] RBAC: Authorizing `identity:delete_user(user_id=c686de5175df424382c6de5ffe1dc256)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:25.860574 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b7f24-ee79-4c5a-8fc3-ddb7c1d4caee admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:25.914367 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-691b7f24-ee79-4c5a-8fc3-ddb7c1d4caee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c686de5175df424382c6de5ffe1dc256'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:25.921679 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-691b7f24-ee79-4c5a-8fc3-ddb7c1d4caee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c686de5175df424382c6de5ffe1dc256'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:25.922771 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-691b7f24-ee79-4c5a-8fc3-ddb7c1d4caee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c686de5175df424382c6de5ffe1dc256'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:25.934780 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-691b7f24-ee79-4c5a-8fc3-ddb7c1d4caee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c686de5175df424382c6de5ffe1dc256'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:25.935544 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-691b7f24-ee79-4c5a-8fc3-ddb7c1d4caee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c686de5175df424382c6de5ffe1dc256'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:25.953900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1430/2874] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:35:25 2022] DELETE /identity/v3/users/c686de5175df424382c6de5ffe1dc256 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:35:26.237672 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd85b78-a010-468c-95ea-634b16394dec admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:26.239280 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-dfd85b78-a010-468c-95ea-634b16394dec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:35:26.252225 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd85b78-a010-468c-95ea-634b16394dec admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:26.297148 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:35:26.312025 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fc51e8-c78e-40dd-8855-ea4fc993fd02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:26.312686 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc51e8-c78e-40dd-8855-ea4fc993fd02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:26.312959 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc51e8-c78e-40dd-8855-ea4fc993fd02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:26.313222 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc51e8-c78e-40dd-8855-ea4fc993fd02 admin admin] PATH_INFO: `/v3/projects/1c2e8b25b468405a93b70e34294d978b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:26.314344 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fc51e8-c78e-40dd-8855-ea4fc993fd02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c2e8b25b468405a93b70e34294d978b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:26.316264 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fc51e8-c78e-40dd-8855-ea4fc993fd02 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:26.365485 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e5fc51e8-c78e-40dd-8855-ea4fc993fd02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c2e8b25b468405a93b70e34294d978b', 'request_id': 'req-e5fc51e8-c78e-40dd-8855-ea4fc993fd02'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:26.371733 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e5fc51e8-c78e-40dd-8855-ea4fc993fd02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c2e8b25b468405a93b70e34294d978b', 'request_id': 'req-e5fc51e8-c78e-40dd-8855-ea4fc993fd02'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:26.381566 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e5fc51e8-c78e-40dd-8855-ea4fc993fd02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c2e8b25b468405a93b70e34294d978b', 'request_id': 'req-e5fc51e8-c78e-40dd-8855-ea4fc993fd02'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:26.394793 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e5fc51e8-c78e-40dd-8855-ea4fc993fd02 admin admin] The token cache is being invalidate because project 1c2e8b25b468405a93b70e34294d978b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:35:26.395077 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e5fc51e8-c78e-40dd-8855-ea4fc993fd02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:26.395737 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e5fc51e8-c78e-40dd-8855-ea4fc993fd02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:35:26.397233 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1445/2875] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:35:26 2022] DELETE /identity/v3/projects/1c2e8b25b468405a93b70e34294d978b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:35:26.402296 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619493c-56dd-4069-8752-28001b4f3675 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:26.402583 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619493c-56dd-4069-8752-28001b4f3675 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:26.402852 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619493c-56dd-4069-8752-28001b4f3675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:26.424201 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-9619493c-56dd-4069-8752-28001b4f3675 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:35:26.425388 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-9619493c-56dd-4069-8752-28001b4f3675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:35:26.446021 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:35:26.485654 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1431/2876] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:35:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:35:26.493728 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a56d8-4b19-481d-b537-a623f6ed159f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:26.494033 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a56d8-4b19-481d-b537-a623f6ed159f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:26.494330 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a56d8-4b19-481d-b537-a623f6ed159f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:26.515897 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-2f8a56d8-4b19-481d-b537-a623f6ed159f None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:35:26.517708 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2f8a56d8-4b19-481d-b537-a623f6ed159f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:35:26.531339 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1446/2877] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:35:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:35:26.537061 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9619493c-56dd-4069-8752-28001b4f3675 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:26.546925 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9619493c-56dd-4069-8752-28001b4f3675 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:26.552158 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fd4b23-4129-445d-aaf7-6d8c9f2de4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:26.552751 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd4b23-4129-445d-aaf7-6d8c9f2de4cd admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:26.553026 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd4b23-4129-445d-aaf7-6d8c9f2de4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:26.553286 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd4b23-4129-445d-aaf7-6d8c9f2de4cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:26.553953 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fd4b23-4129-445d-aaf7-6d8c9f2de4cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:35:26.554273 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fd4b23-4129-445d-aaf7-6d8c9f2de4cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:26.555872 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fd4b23-4129-445d-aaf7-6d8c9f2de4cd admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:26.567589 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1432/2878] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:35:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:35:26.573835 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a56d8-4b19-481d-b537-a623f6ed159f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:26.583541 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a56d8-4b19-481d-b537-a623f6ed159f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:26.588815 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f333c9-7424-49ee-9a57-844d0bdc48f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:26.589544 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f333c9-7424-49ee-9a57-844d0bdc48f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:26.589826 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f333c9-7424-49ee-9a57-844d0bdc48f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:26.590146 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f333c9-7424-49ee-9a57-844d0bdc48f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:26.590782 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f333c9-7424-49ee-9a57-844d0bdc48f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:26.592696 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f333c9-7424-49ee-9a57-844d0bdc48f7 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:26.623338 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1447/2879] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:35:26 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:35:26.629754 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fd4b23-4129-445d-aaf7-6d8c9f2de4cd admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:26.641073 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fd4b23-4129-445d-aaf7-6d8c9f2de4cd admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:26.699326 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09503507-0f6c-4512-bb04-a3d3fc20224d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:26.699931 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09503507-0f6c-4512-bb04-a3d3fc20224d admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:26.700208 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09503507-0f6c-4512-bb04-a3d3fc20224d admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:26.700477 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09503507-0f6c-4512-bb04-a3d3fc20224d admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:26.701109 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09503507-0f6c-4512-bb04-a3d3fc20224d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:26.703104 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09503507-0f6c-4512-bb04-a3d3fc20224d admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:26.727280 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1433/2880] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:35:26 2022] POST /identity/v3/users => generated 283 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:35:26.732650 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f333c9-7424-49ee-9a57-844d0bdc48f7 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:26.747712 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f333c9-7424-49ee-9a57-844d0bdc48f7 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:26.753000 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3b85c2-c759-421e-9c36-c3fba56403dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:26.753595 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b85c2-c759-421e-9c36-c3fba56403dd admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:26.753877 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b85c2-c759-421e-9c36-c3fba56403dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:26.754178 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b85c2-c759-421e-9c36-c3fba56403dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:26.754825 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3b85c2-c759-421e-9c36-c3fba56403dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:26.756421 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3b85c2-c759-421e-9c36-c3fba56403dd admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:26.766723 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1448/2881] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:35:26.770759 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09503507-0f6c-4512-bb04-a3d3fc20224d admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:26.781640 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09503507-0f6c-4512-bb04-a3d3fc20224d admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:26.787127 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdc8c50-697b-4db6-94b3-f7a7640f8e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:26.787727 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc8c50-697b-4db6-94b3-f7a7640f8e09 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:26.788008 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc8c50-697b-4db6-94b3-f7a7640f8e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:26.788343 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc8c50-697b-4db6-94b3-f7a7640f8e09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:26.789025 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdc8c50-697b-4db6-94b3-f7a7640f8e09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:26.790674 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdc8c50-697b-4db6-94b3-f7a7640f8e09 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:26.802838 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1434/2882] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:35:26.807107 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3b85c2-c759-421e-9c36-c3fba56403dd admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:26.817584 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3b85c2-c759-421e-9c36-c3fba56403dd admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:26.823011 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd0ed0f-658e-4678-bcf7-2103299a8421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:26.823669 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0ed0f-658e-4678-bcf7-2103299a8421 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:26.823943 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0ed0f-658e-4678-bcf7-2103299a8421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:26.824210 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0ed0f-658e-4678-bcf7-2103299a8421 admin admin] PATH_INFO: `/v3/projects/955f9aa2da264d9391403d6eae59d00a/users/578575c1c4cc4db6818f953523ae83d0/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:26.839278 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd0ed0f-658e-4678-bcf7-2103299a8421 admin admin] RBAC: Authorizing `identity:create_grant(project_id=955f9aa2da264d9391403d6eae59d00a, user_id=578575c1c4cc4db6818f953523ae83d0, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:26.841927 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd0ed0f-658e-4678-bcf7-2103299a8421 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:26.857197 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1449/2883] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:35:26 2022] PUT /identity/v3/projects/955f9aa2da264d9391403d6eae59d00a/users/578575c1c4cc4db6818f953523ae83d0/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:35:26.867526 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181cc58-6d5b-45d1-a21b-cc2b73e234ae None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:26.867819 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181cc58-6d5b-45d1-a21b-cc2b73e234ae None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:26.868086 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181cc58-6d5b-45d1-a21b-cc2b73e234ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:26.897873 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-4181cc58-6d5b-45d1-a21b-cc2b73e234ae None None] MFA Rules not processed for user `578575c1c4cc4db6818f953523ae83d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:35:26.899027 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-4181cc58-6d5b-45d1-a21b-cc2b73e234ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:35:26.970917 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1435/2884] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:35:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:35:26.983983 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd0ed0f-658e-4678-bcf7-2103299a8421 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:35:26.985851 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-cbd0ed0f-658e-4678-bcf7-2103299a8421 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:35:27.054626 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd0ed0f-658e-4678-bcf7-2103299a8421 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:35:27.059321 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6573975-ec1f-4520-ad22-ebedbc85eedf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:35:27.059934 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6573975-ec1f-4520-ad22-ebedbc85eedf service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:35:27.060223 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6573975-ec1f-4520-ad22-ebedbc85eedf service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:35:27.060521 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6573975-ec1f-4520-ad22-ebedbc85eedf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:35:27.069870 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6573975-ec1f-4520-ad22-ebedbc85eedf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:35:27.071409 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6573975-ec1f-4520-ad22-ebedbc85eedf service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:35:27.080524 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1450/2885] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:35:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:35:36.968349 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=87,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:35:37.076760 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:36:09.078442 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4181cc58-6d5b-45d1-a21b-cc2b73e234ae None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:36:09.144234 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4181cc58-6d5b-45d1-a21b-cc2b73e234ae None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:36:09.149270 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328310e1-c411-4190-99a2-408ad5bc3ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:36:09.149898 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328310e1-c411-4190-99a2-408ad5bc3ff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:09.150218 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328310e1-c411-4190-99a2-408ad5bc3ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:09.150497 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328310e1-c411-4190-99a2-408ad5bc3ff8 admin admin] PATH_INFO: `/v3/users/578575c1c4cc4db6818f953523ae83d0` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:09.151898 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328310e1-c411-4190-99a2-408ad5bc3ff8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=578575c1c4cc4db6818f953523ae83d0)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:36:09.154384 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328310e1-c411-4190-99a2-408ad5bc3ff8 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:36:09.204164 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-328310e1-c411-4190-99a2-408ad5bc3ff8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '578575c1c4cc4db6818f953523ae83d0'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:36:09.211487 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-328310e1-c411-4190-99a2-408ad5bc3ff8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '578575c1c4cc4db6818f953523ae83d0'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:36:09.212139 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-328310e1-c411-4190-99a2-408ad5bc3ff8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '578575c1c4cc4db6818f953523ae83d0'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:36:09.222627 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-328310e1-c411-4190-99a2-408ad5bc3ff8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '578575c1c4cc4db6818f953523ae83d0'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:36:09.223394 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-328310e1-c411-4190-99a2-408ad5bc3ff8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '578575c1c4cc4db6818f953523ae83d0'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:36:09.241912 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1436/2886] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:36:09 2022] DELETE /identity/v3/users/578575c1c4cc4db6818f953523ae83d0 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:36:09.246592 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeeab61-7516-450e-be8d-58c6bb952ff7 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:09.247123 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeeab61-7516-450e-be8d-58c6bb952ff7 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:09.247123 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeeab61-7516-450e-be8d-58c6bb952ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:09.268902 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-7eeeab61-7516-450e-be8d-58c6bb952ff7 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:36:09.270113 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7eeeab61-7516-450e-be8d-58c6bb952ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:36:09.336798 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1451/2887] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:36:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:36:09.349072 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328310e1-c411-4190-99a2-408ad5bc3ff8 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:36:09.350763 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-328310e1-c411-4190-99a2-408ad5bc3ff8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:36:09.409859 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328310e1-c411-4190-99a2-408ad5bc3ff8 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:36:09.414399 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce1f35a-cc34-4498-b600-9b63a4606297 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:36:09.415004 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1f35a-cc34-4498-b600-9b63a4606297 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:09.415281 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1f35a-cc34-4498-b600-9b63a4606297 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:09.415544 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1f35a-cc34-4498-b600-9b63a4606297 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:09.425574 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce1f35a-cc34-4498-b600-9b63a4606297 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:36:09.427895 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce1f35a-cc34-4498-b600-9b63a4606297 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:36:09.438441 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1437/2888] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:36:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:36:09.724258 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeeab61-7516-450e-be8d-58c6bb952ff7 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:36:09.725680 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7eeeab61-7516-450e-be8d-58c6bb952ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:36:09.737535 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeeab61-7516-450e-be8d-58c6bb952ff7 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:36:09.742355 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f6dda3-e2c6-4fc3-8f31-3bfd25b1171a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:36:09.742994 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6dda3-e2c6-4fc3-8f31-3bfd25b1171a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:09.743267 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6dda3-e2c6-4fc3-8f31-3bfd25b1171a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:09.743531 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6dda3-e2c6-4fc3-8f31-3bfd25b1171a admin admin] PATH_INFO: `/v3/projects/955f9aa2da264d9391403d6eae59d00a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:09.744669 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f6dda3-e2c6-4fc3-8f31-3bfd25b1171a admin admin] RBAC: Authorizing `identity:delete_project(project_id=955f9aa2da264d9391403d6eae59d00a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:36:09.746609 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f6dda3-e2c6-4fc3-8f31-3bfd25b1171a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:36:09.796668 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-70f6dda3-e2c6-4fc3-8f31-3bfd25b1171a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '955f9aa2da264d9391403d6eae59d00a', 'request_id': 'req-70f6dda3-e2c6-4fc3-8f31-3bfd25b1171a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:36:09.803076 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-70f6dda3-e2c6-4fc3-8f31-3bfd25b1171a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '955f9aa2da264d9391403d6eae59d00a', 'request_id': 'req-70f6dda3-e2c6-4fc3-8f31-3bfd25b1171a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:36:09.812487 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-70f6dda3-e2c6-4fc3-8f31-3bfd25b1171a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '955f9aa2da264d9391403d6eae59d00a', 'request_id': 'req-70f6dda3-e2c6-4fc3-8f31-3bfd25b1171a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:36:09.823313 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-70f6dda3-e2c6-4fc3-8f31-3bfd25b1171a admin admin] The token cache is being invalidate because project 955f9aa2da264d9391403d6eae59d00a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:36:09.823620 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-70f6dda3-e2c6-4fc3-8f31-3bfd25b1171a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:36:09.824234 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-70f6dda3-e2c6-4fc3-8f31-3bfd25b1171a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:36:09.825742 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1452/2889] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:36:09 2022] DELETE /identity/v3/projects/955f9aa2da264d9391403d6eae59d00a => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:36:09.830833 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ababf02-a93d-4507-bc52-3cf46d7a3772 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:09.831111 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ababf02-a93d-4507-bc52-3cf46d7a3772 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:09.831372 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ababf02-a93d-4507-bc52-3cf46d7a3772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:09.852178 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-8ababf02-a93d-4507-bc52-3cf46d7a3772 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:36:09.853348 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-8ababf02-a93d-4507-bc52-3cf46d7a3772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:36:09.870700 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:36:09.913611 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1438/2890] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:36:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:36:09.921839 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388762f-0a22-4613-b87d-17d7eb92babd None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:09.922164 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388762f-0a22-4613-b87d-17d7eb92babd None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:09.922441 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388762f-0a22-4613-b87d-17d7eb92babd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:09.942808 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-3388762f-0a22-4613-b87d-17d7eb92babd None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:36:09.944007 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3388762f-0a22-4613-b87d-17d7eb92babd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:36:09.952780 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1453/2891] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:36:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:36:09.957565 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ababf02-a93d-4507-bc52-3cf46d7a3772 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:36:09.972676 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ababf02-a93d-4507-bc52-3cf46d7a3772 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:36:09.979750 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e2a5c6-c83b-4602-a92f-29d2a39cf60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:36:09.980349 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2a5c6-c83b-4602-a92f-29d2a39cf60c admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:09.980624 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2a5c6-c83b-4602-a92f-29d2a39cf60c admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:09.980865 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2a5c6-c83b-4602-a92f-29d2a39cf60c admin admin] PATH_INFO: `/v3/domains` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:09.981510 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e2a5c6-c83b-4602-a92f-29d2a39cf60c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:36:09.981804 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e2a5c6-c83b-4602-a92f-29d2a39cf60c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:36:09.983527 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e2a5c6-c83b-4602-a92f-29d2a39cf60c admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:36:09.995588 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1439/2892] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:36:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 05:36:10.000230 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388762f-0a22-4613-b87d-17d7eb92babd None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:36:10.013966 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388762f-0a22-4613-b87d-17d7eb92babd None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:36:10.019433 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6177c5-a465-4dab-b6e3-0856f56ddee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:36:10.020038 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6177c5-a465-4dab-b6e3-0856f56ddee6 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:10.020313 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6177c5-a465-4dab-b6e3-0856f56ddee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:10.020581 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6177c5-a465-4dab-b6e3-0856f56ddee6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:10.021200 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6177c5-a465-4dab-b6e3-0856f56ddee6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:36:10.023125 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6177c5-a465-4dab-b6e3-0856f56ddee6 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:36:10.053121 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1454/2893] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:36:09 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:36:10.057503 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e2a5c6-c83b-4602-a92f-29d2a39cf60c admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:36:10.067756 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e2a5c6-c83b-4602-a92f-29d2a39cf60c admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:36:10.124699 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f71713-93ad-4d0d-b4f5-26228878781e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:36:10.125363 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f71713-93ad-4d0d-b4f5-26228878781e admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:10.125554 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f71713-93ad-4d0d-b4f5-26228878781e admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:10.125820 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f71713-93ad-4d0d-b4f5-26228878781e admin admin] PATH_INFO: `/v3/users` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:10.126473 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f71713-93ad-4d0d-b4f5-26228878781e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:36:10.128397 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f71713-93ad-4d0d-b4f5-26228878781e admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:36:10.151875 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1440/2894] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:36:10 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:36:10.156065 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6177c5-a465-4dab-b6e3-0856f56ddee6 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:36:10.166481 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6177c5-a465-4dab-b6e3-0856f56ddee6 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:36:10.171225 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f3e692-d11b-4a02-881a-e51be35c965c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:36:10.171824 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3e692-d11b-4a02-881a-e51be35c965c admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:10.172097 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3e692-d11b-4a02-881a-e51be35c965c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:10.172359 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3e692-d11b-4a02-881a-e51be35c965c admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:10.172993 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f3e692-d11b-4a02-881a-e51be35c965c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:36:10.174614 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f3e692-d11b-4a02-881a-e51be35c965c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:36:10.184548 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1455/2895] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:36:10.188451 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f71713-93ad-4d0d-b4f5-26228878781e admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:36:10.198747 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f71713-93ad-4d0d-b4f5-26228878781e admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:36:10.203499 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90288868-ecf6-4b1a-b5f6-b071ed8e56da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:36:10.204114 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90288868-ecf6-4b1a-b5f6-b071ed8e56da admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:10.204388 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90288868-ecf6-4b1a-b5f6-b071ed8e56da admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:10.204657 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90288868-ecf6-4b1a-b5f6-b071ed8e56da admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:10.205300 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90288868-ecf6-4b1a-b5f6-b071ed8e56da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:36:10.206951 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90288868-ecf6-4b1a-b5f6-b071ed8e56da admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:36:10.217754 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1441/2896] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 05:36:10.221478 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f3e692-d11b-4a02-881a-e51be35c965c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:36:10.231603 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f3e692-d11b-4a02-881a-e51be35c965c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:36:10.236550 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41049cb1-9dc1-4af0-9c2f-c54ce155598f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:36:10.237185 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41049cb1-9dc1-4af0-9c2f-c54ce155598f admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:10.237460 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41049cb1-9dc1-4af0-9c2f-c54ce155598f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:10.237728 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41049cb1-9dc1-4af0-9c2f-c54ce155598f admin admin] PATH_INFO: `/v3/projects/6ee67d32b3cf47f9b77e5a00fcd88188/users/5815ac04238a4843860f7a6380887afd/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:10.251946 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41049cb1-9dc1-4af0-9c2f-c54ce155598f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ee67d32b3cf47f9b77e5a00fcd88188, user_id=5815ac04238a4843860f7a6380887afd, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:36:10.254604 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41049cb1-9dc1-4af0-9c2f-c54ce155598f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:36:10.268489 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1456/2897] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:36:10 2022] PUT /identity/v3/projects/6ee67d32b3cf47f9b77e5a00fcd88188/users/5815ac04238a4843860f7a6380887afd/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:36:10.280493 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dba498-b36c-4d05-b4d8-6818a142cb6e None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:10.280777 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dba498-b36c-4d05-b4d8-6818a142cb6e None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:10.281072 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dba498-b36c-4d05-b4d8-6818a142cb6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:10.309370 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-f4dba498-b36c-4d05-b4d8-6818a142cb6e None None] MFA Rules not processed for user `5815ac04238a4843860f7a6380887afd`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:36:10.310523 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-f4dba498-b36c-4d05-b4d8-6818a142cb6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:36:10.376384 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1442/2898] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:36:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:36:10.387413 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41049cb1-9dc1-4af0-9c2f-c54ce155598f admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:36:10.389325 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-41049cb1-9dc1-4af0-9c2f-c54ce155598f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:36:10.411984 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:36:10.449010 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41049cb1-9dc1-4af0-9c2f-c54ce155598f admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:36:10.453047 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8fac2-4fc3-4f30-8702-e7c9390856a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:36:10.453645 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8fac2-4fc3-4f30-8702-e7c9390856a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:10.453922 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8fac2-4fc3-4f30-8702-e7c9390856a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:10.454247 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8fac2-4fc3-4f30-8702-e7c9390856a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:10.462801 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b8fac2-4fc3-4f30-8702-e7c9390856a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:36:10.464271 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b8fac2-4fc3-4f30-8702-e7c9390856a2 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:36:10.476922 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1457/2899] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:36:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:36:11.867271 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b985f291-cfe9-44cf-a98b-af299362c29d None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:11.867826 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b985f291-cfe9-44cf-a98b-af299362c29d None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:11.867826 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b985f291-cfe9-44cf-a98b-af299362c29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:11.892036 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-b985f291-cfe9-44cf-a98b-af299362c29d None None] MFA Rules not processed for user `5815ac04238a4843860f7a6380887afd`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:36:11.893545 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b985f291-cfe9-44cf-a98b-af299362c29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:36:11.903712 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1443/2900] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:36:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:36:12.386036 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8fac2-4fc3-4f30-8702-e7c9390856a2 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:36:12.387563 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-00b8fac2-4fc3-4f30-8702-e7c9390856a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:36:12.458758 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8fac2-4fc3-4f30-8702-e7c9390856a2 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:36:12.464132 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f62917b-5cfd-4a13-b15b-cbe2281de5b0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd4387035099416789009f0ecb8ad282', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:36:12.464739 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f62917b-5cfd-4a13-b15b-cbe2281de5b0 service placement] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:12.465014 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f62917b-5cfd-4a13-b15b-cbe2281de5b0 service placement] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:12.465280 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f62917b-5cfd-4a13-b15b-cbe2281de5b0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:12.467046 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2f62917b-5cfd-4a13-b15b-cbe2281de5b0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:36:12.478439 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f62917b-5cfd-4a13-b15b-cbe2281de5b0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:36:12.480384 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f62917b-5cfd-4a13-b15b-cbe2281de5b0 service placement] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:36:12.490041 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1458/2901] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:36:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:36:14.423102 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b985f291-cfe9-44cf-a98b-af299362c29d None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:36:14.424690 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b985f291-cfe9-44cf-a98b-af299362c29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:36:14.438975 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:36:14.486307 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b985f291-cfe9-44cf-a98b-af299362c29d None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:36:14.489987 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02c20fd-630e-4856-938f-67d308992dfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:36:14.490623 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c20fd-630e-4856-938f-67d308992dfe service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:14.490893 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c20fd-630e-4856-938f-67d308992dfe service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:14.491157 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c20fd-630e-4856-938f-67d308992dfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:14.492880 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a02c20fd-630e-4856-938f-67d308992dfe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:36:14.502910 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02c20fd-630e-4856-938f-67d308992dfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:36:14.504342 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02c20fd-630e-4856-938f-67d308992dfe service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:36:14.512439 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1444/2902] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:36:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:36:15.440505 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f62917b-5cfd-4a13-b15b-cbe2281de5b0 service placement] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:36:15.442250 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2f62917b-5cfd-4a13-b15b-cbe2281de5b0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:36:15.521496 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f62917b-5cfd-4a13-b15b-cbe2281de5b0 service placement] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:36:15.527196 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3127105c-de97-4eff-8ac6-ac762aadea72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:36:15.527811 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127105c-de97-4eff-8ac6-ac762aadea72 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:15.528090 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127105c-de97-4eff-8ac6-ac762aadea72 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:15.528739 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127105c-de97-4eff-8ac6-ac762aadea72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:15.530258 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3127105c-de97-4eff-8ac6-ac762aadea72 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:36:15.543578 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3127105c-de97-4eff-8ac6-ac762aadea72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:36:15.545188 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3127105c-de97-4eff-8ac6-ac762aadea72 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:36:15.556147 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1459/2903] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:36:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:36:20.335578 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02c20fd-630e-4856-938f-67d308992dfe service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:36:20.337206 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-a02c20fd-630e-4856-938f-67d308992dfe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:36:20.353204 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02c20fd-630e-4856-938f-67d308992dfe service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:36:20.359505 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dafb4d-b542-4644-8c3c-ccb75d6b559c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:36:20.360188 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dafb4d-b542-4644-8c3c-ccb75d6b559c service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:20.360470 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dafb4d-b542-4644-8c3c-ccb75d6b559c service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:20.360729 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dafb4d-b542-4644-8c3c-ccb75d6b559c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:20.362538 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-27dafb4d-b542-4644-8c3c-ccb75d6b559c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:36:20.374984 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dafb4d-b542-4644-8c3c-ccb75d6b559c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:36:20.376773 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dafb4d-b542-4644-8c3c-ccb75d6b559c service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:36:20.388010 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1445/2904] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:36:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:36:25.551402 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=83 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:36:27.258155 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25b298-198d-4c28-a8fc-d12334730145 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:27.259199 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25b298-198d-4c28-a8fc-d12334730145 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:27.259199 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25b298-198d-4c28-a8fc-d12334730145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:27.315117 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-dd25b298-198d-4c28-a8fc-d12334730145 None None] MFA Rules not processed for user `09bf6db88ce8494a937211a814c9ff84`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:36:27.316889 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-dd25b298-198d-4c28-a8fc-d12334730145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:36:27.392837 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1460/2905] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Aug 31 05:36:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:36:27.442596 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dafb4d-b542-4644-8c3c-ccb75d6b559c service nova] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:36:27.444256 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-27dafb4d-b542-4644-8c3c-ccb75d6b559c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:36:27.539844 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dafb4d-b542-4644-8c3c-ccb75d6b559c service nova] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:36:27.545915 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575d8d31-b954-48b4-8706-2f5aea184ae0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9b82080a73d4c4a8ec08b0e67d6d11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:36:27.546660 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d8d31-b954-48b4-8706-2f5aea184ae0 service swift] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:27.546960 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d8d31-b954-48b4-8706-2f5aea184ae0 service swift] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:27.547225 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d8d31-b954-48b4-8706-2f5aea184ae0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:27.556709 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575d8d31-b954-48b4-8706-2f5aea184ae0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:36:27.558428 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575d8d31-b954-48b4-8706-2f5aea184ae0 service swift] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:36:27.567011 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1446/2906] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Aug 31 05:36:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 05:36:28.095477 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f2979-47fc-4a3b-a793-fc30910a84bc None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:28.095764 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f2979-47fc-4a3b-a793-fc30910a84bc None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:28.096038 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f2979-47fc-4a3b-a793-fc30910a84bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:28.117002 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-299f2979-47fc-4a3b-a793-fc30910a84bc None None] MFA Rules not processed for user `09bf6db88ce8494a937211a814c9ff84`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:36:28.118227 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-299f2979-47fc-4a3b-a793-fc30910a84bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:36:28.126131 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1461/2907] 10.222.0.36 () {58 vars in 967 bytes} [Wed Aug 31 05:36:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:36:28.138424 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575d8d31-b954-48b4-8706-2f5aea184ae0 service swift] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:36:28.149759 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575d8d31-b954-48b4-8706-2f5aea184ae0 service swift] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:36:28.156282 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26324508-54ce-4fa6-bfcd-64a0fed59048 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9b82080a73d4c4a8ec08b0e67d6d11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:36:28.156973 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26324508-54ce-4fa6-bfcd-64a0fed59048 service swift] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:28.157248 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26324508-54ce-4fa6-bfcd-64a0fed59048 service swift] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:28.157513 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26324508-54ce-4fa6-bfcd-64a0fed59048 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:28.165853 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26324508-54ce-4fa6-bfcd-64a0fed59048 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:36:28.167477 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26324508-54ce-4fa6-bfcd-64a0fed59048 service swift] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:36:28.175683 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1447/2908] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Aug 31 05:36:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 05:36:28.406486 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865be8b-ce2a-4c1f-9d58-c1c80459c86b None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:28.406996 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865be8b-ce2a-4c1f-9d58-c1c80459c86b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:28.406996 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865be8b-ce2a-4c1f-9d58-c1c80459c86b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:28.426179 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-9865be8b-ce2a-4c1f-9d58-c1c80459c86b None None] MFA Rules not processed for user `09bf6db88ce8494a937211a814c9ff84`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:36:28.427353 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9865be8b-ce2a-4c1f-9d58-c1c80459c86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:36:28.434801 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1462/2909] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Aug 31 05:36:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:36:28.446700 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26324508-54ce-4fa6-bfcd-64a0fed59048 service swift] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:36:28.457954 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26324508-54ce-4fa6-bfcd-64a0fed59048 service swift] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:36:28.464412 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d50ae0e-6c27-4d94-80f8-cb0b602859e2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9b82080a73d4c4a8ec08b0e67d6d11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:36:28.465042 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50ae0e-6c27-4d94-80f8-cb0b602859e2 service swift] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:28.465321 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50ae0e-6c27-4d94-80f8-cb0b602859e2 service swift] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:28.465588 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50ae0e-6c27-4d94-80f8-cb0b602859e2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:28.474050 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d50ae0e-6c27-4d94-80f8-cb0b602859e2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:36:28.475638 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d50ae0e-6c27-4d94-80f8-cb0b602859e2 service swift] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:36:28.484169 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1448/2910] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Aug 31 05:36:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 05:36:28.808046 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a417b4e2-cd48-43bf-92dd-6fa344ebef80 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:28.808330 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a417b4e2-cd48-43bf-92dd-6fa344ebef80 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:28.808686 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a417b4e2-cd48-43bf-92dd-6fa344ebef80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:28.830695 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a417b4e2-cd48-43bf-92dd-6fa344ebef80 None None] MFA Rules not processed for user `09bf6db88ce8494a937211a814c9ff84`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:36:28.831909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a417b4e2-cd48-43bf-92dd-6fa344ebef80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:36:28.840322 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1463/2911] 10.222.0.36 () {58 vars in 967 bytes} [Wed Aug 31 05:36:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:36:28.855980 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d50ae0e-6c27-4d94-80f8-cb0b602859e2 service swift] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:36:28.866955 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d50ae0e-6c27-4d94-80f8-cb0b602859e2 service swift] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:36:28.872106 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b2c9d-1d06-4c35-986d-4aa4efd65fcf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9b82080a73d4c4a8ec08b0e67d6d11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:36:28.872710 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b2c9d-1d06-4c35-986d-4aa4efd65fcf service swift] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:28.872985 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b2c9d-1d06-4c35-986d-4aa4efd65fcf service swift] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:28.873371 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b2c9d-1d06-4c35-986d-4aa4efd65fcf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:28.881791 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8b2c9d-1d06-4c35-986d-4aa4efd65fcf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:36:28.883556 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8b2c9d-1d06-4c35-986d-4aa4efd65fcf service swift] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:36:28.892319 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1449/2912] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Aug 31 05:36:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 05:36:30.997884 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a417b4e2-cd48-43bf-92dd-6fa344ebef80 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:36:31.076607 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a417b4e2-cd48-43bf-92dd-6fa344ebef80 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:36:31.082513 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3de835-2db5-4274-8b2f-b753a84d6805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:36:31.083143 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3de835-2db5-4274-8b2f-b753a84d6805 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:31.083419 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3de835-2db5-4274-8b2f-b753a84d6805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:31.083675 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3de835-2db5-4274-8b2f-b753a84d6805 admin admin] PATH_INFO: `/v3/users/5815ac04238a4843860f7a6380887afd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:31.085013 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3de835-2db5-4274-8b2f-b753a84d6805 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5815ac04238a4843860f7a6380887afd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:36:31.086980 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3de835-2db5-4274-8b2f-b753a84d6805 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:36:31.148175 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ad3de835-2db5-4274-8b2f-b753a84d6805 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5815ac04238a4843860f7a6380887afd'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:36:31.153636 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:36:31.155982 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ad3de835-2db5-4274-8b2f-b753a84d6805 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5815ac04238a4843860f7a6380887afd'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:36:31.163968 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ad3de835-2db5-4274-8b2f-b753a84d6805 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5815ac04238a4843860f7a6380887afd'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:36:31.164550 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ad3de835-2db5-4274-8b2f-b753a84d6805 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5815ac04238a4843860f7a6380887afd'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:36:31.176788 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ad3de835-2db5-4274-8b2f-b753a84d6805 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5815ac04238a4843860f7a6380887afd'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:36:31.182726 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1464/2913] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:36:30 2022] DELETE /identity/v3/users/5815ac04238a4843860f7a6380887afd => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:36:31.187359 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f00e0d0-1baf-45c9-8e99-681835a65084 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:31.187876 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f00e0d0-1baf-45c9-8e99-681835a65084 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:31.187876 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f00e0d0-1baf-45c9-8e99-681835a65084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:31.207218 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-6f00e0d0-1baf-45c9-8e99-681835a65084 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:36:31.208420 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-6f00e0d0-1baf-45c9-8e99-681835a65084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:36:31.234692 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=100 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:36:31.268061 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1450/2914] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:36:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:36:31.278130 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3de835-2db5-4274-8b2f-b753a84d6805 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:36:31.279721 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ad3de835-2db5-4274-8b2f-b753a84d6805 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:36:31.338521 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3de835-2db5-4274-8b2f-b753a84d6805 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:36:31.342767 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f24497b-f776-4afd-a70b-6d79ac0372ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:36:31.343374 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f24497b-f776-4afd-a70b-6d79ac0372ec service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:31.343647 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f24497b-f776-4afd-a70b-6d79ac0372ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:31.343911 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f24497b-f776-4afd-a70b-6d79ac0372ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:31.353113 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f24497b-f776-4afd-a70b-6d79ac0372ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:36:31.354947 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f24497b-f776-4afd-a70b-6d79ac0372ec service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:36:31.364943 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1465/2915] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:36:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:36:31.543315 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f00e0d0-1baf-45c9-8e99-681835a65084 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:36:31.544853 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-6f00e0d0-1baf-45c9-8e99-681835a65084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:36:31.560859 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f00e0d0-1baf-45c9-8e99-681835a65084 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:36:31.566064 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13f81b6-fe42-4865-b97a-53113d2b2b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:36:31.566726 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f81b6-fe42-4865-b97a-53113d2b2b8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:31.567061 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f81b6-fe42-4865-b97a-53113d2b2b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:31.567240 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f81b6-fe42-4865-b97a-53113d2b2b8b admin admin] PATH_INFO: `/v3/projects/6ee67d32b3cf47f9b77e5a00fcd88188` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:31.568310 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13f81b6-fe42-4865-b97a-53113d2b2b8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ee67d32b3cf47f9b77e5a00fcd88188)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:36:31.570358 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13f81b6-fe42-4865-b97a-53113d2b2b8b admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:36:31.620749 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c13f81b6-fe42-4865-b97a-53113d2b2b8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ee67d32b3cf47f9b77e5a00fcd88188', 'request_id': 'req-c13f81b6-fe42-4865-b97a-53113d2b2b8b'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:36:31.627114 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c13f81b6-fe42-4865-b97a-53113d2b2b8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ee67d32b3cf47f9b77e5a00fcd88188', 'request_id': 'req-c13f81b6-fe42-4865-b97a-53113d2b2b8b'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:36:31.633391 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c13f81b6-fe42-4865-b97a-53113d2b2b8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ee67d32b3cf47f9b77e5a00fcd88188', 'request_id': 'req-c13f81b6-fe42-4865-b97a-53113d2b2b8b'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:36:31.649451 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c13f81b6-fe42-4865-b97a-53113d2b2b8b admin admin] The token cache is being invalidate because project 6ee67d32b3cf47f9b77e5a00fcd88188 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:36:31.650214 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c13f81b6-fe42-4865-b97a-53113d2b2b8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:36:31.650939 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c13f81b6-fe42-4865-b97a-53113d2b2b8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:36:31.652532 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1451/2916] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:36:31 2022] DELETE /identity/v3/projects/6ee67d32b3cf47f9b77e5a00fcd88188 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:36:31.657901 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1110662-cf55-45bc-9aec-d9cc37402a00 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:31.658218 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1110662-cf55-45bc-9aec-d9cc37402a00 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:31.658482 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1110662-cf55-45bc-9aec-d9cc37402a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:31.678725 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f1110662-cf55-45bc-9aec-d9cc37402a00 None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:36:31.679938 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f1110662-cf55-45bc-9aec-d9cc37402a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:36:31.742217 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1466/2917] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:36:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:36:31.751998 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3247c2-d4b0-4930-aab0-f3f0f6aa617c None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:31.752438 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3247c2-d4b0-4930-aab0-f3f0f6aa617c None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:31.752845 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3247c2-d4b0-4930-aab0-f3f0f6aa617c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:31.777314 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-0e3247c2-d4b0-4930-aab0-f3f0f6aa617c None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:36:31.778628 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-0e3247c2-d4b0-4930-aab0-f3f0f6aa617c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:36:31.787641 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1452/2918] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:36:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:36:31.791951 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1110662-cf55-45bc-9aec-d9cc37402a00 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:36:31.801816 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1110662-cf55-45bc-9aec-d9cc37402a00 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:36:31.807502 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c557b-ba04-4e4f-ba08-f30ace04f243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:36:31.808378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c557b-ba04-4e4f-ba08-f30ace04f243 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:31.808723 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c557b-ba04-4e4f-ba08-f30ace04f243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:31.809069 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c557b-ba04-4e4f-ba08-f30ace04f243 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:31.809850 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177c557b-ba04-4e4f-ba08-f30ace04f243 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 05:36:31.810178 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177c557b-ba04-4e4f-ba08-f30ace04f243 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:36:31.811782 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177c557b-ba04-4e4f-ba08-f30ace04f243 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:36:31.823985 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1467/2919] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Aug 31 05:36:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 05:36:31.828363 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3247c2-d4b0-4930-aab0-f3f0f6aa617c None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:36:31.838136 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3247c2-d4b0-4930-aab0-f3f0f6aa617c None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:36:31.844179 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a70ec1d-dd50-4e90-8334-4aeae065859c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:36:31.844776 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70ec1d-dd50-4e90-8334-4aeae065859c admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:31.845045 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70ec1d-dd50-4e90-8334-4aeae065859c admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:31.845307 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70ec1d-dd50-4e90-8334-4aeae065859c admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:31.845941 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a70ec1d-dd50-4e90-8334-4aeae065859c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:36:31.847883 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a70ec1d-dd50-4e90-8334-4aeae065859c admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:36:31.878712 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1453/2920] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:36:31 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:36:31.883107 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c557b-ba04-4e4f-ba08-f30ace04f243 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:36:31.893115 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c557b-ba04-4e4f-ba08-f30ace04f243 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:36:31.948549 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a9d20-bb29-4c10-9b98-b1016ee8c073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:36:31.949152 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a9d20-bb29-4c10-9b98-b1016ee8c073 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:31.949422 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a9d20-bb29-4c10-9b98-b1016ee8c073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:31.949685 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a9d20-bb29-4c10-9b98-b1016ee8c073 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:31.950350 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310a9d20-bb29-4c10-9b98-b1016ee8c073 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:36:31.952295 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310a9d20-bb29-4c10-9b98-b1016ee8c073 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:36:31.961615 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:36:31.976171 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1468/2921] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:36:31 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:36:31.980791 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a70ec1d-dd50-4e90-8334-4aeae065859c admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:36:31.991270 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a70ec1d-dd50-4e90-8334-4aeae065859c admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:36:31.996590 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f9eda6-0446-4a37-9c33-78fdfcb940f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:36:31.997200 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9eda6-0446-4a37-9c33-78fdfcb940f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:31.997477 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9eda6-0446-4a37-9c33-78fdfcb940f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:31.997745 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9eda6-0446-4a37-9c33-78fdfcb940f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:31.998440 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f9eda6-0446-4a37-9c33-78fdfcb940f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:36:32.000054 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f9eda6-0446-4a37-9c33-78fdfcb940f0 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:36:32.010393 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1454/2922] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:36:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:36:32.014424 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a9d20-bb29-4c10-9b98-b1016ee8c073 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:36:32.024386 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a9d20-bb29-4c10-9b98-b1016ee8c073 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:36:32.029314 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de1aa7-1281-4da9-ba37-d232ebefaf25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:36:32.029895 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de1aa7-1281-4da9-ba37-d232ebefaf25 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:32.030212 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de1aa7-1281-4da9-ba37-d232ebefaf25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:32.030480 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de1aa7-1281-4da9-ba37-d232ebefaf25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:32.031126 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de1aa7-1281-4da9-ba37-d232ebefaf25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:36:32.032757 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de1aa7-1281-4da9-ba37-d232ebefaf25 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:36:32.045947 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1469/2923] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:36:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:36:32.050615 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f9eda6-0446-4a37-9c33-78fdfcb940f0 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:36:32.061578 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f9eda6-0446-4a37-9c33-78fdfcb940f0 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:36:32.067423 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f771267b-aa52-48d8-b657-8063528caba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:36:32.068076 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771267b-aa52-48d8-b657-8063528caba1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:32.068353 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771267b-aa52-48d8-b657-8063528caba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:32.068620 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771267b-aa52-48d8-b657-8063528caba1 admin admin] PATH_INFO: `/v3/projects/8c2ff14d7aca45419054434365802dbf/users/8ae25b465a47404a80b7f40580e70884/roles/e3235eda7c654e2abf96ed6a494f5152` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:32.083630 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f771267b-aa52-48d8-b657-8063528caba1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c2ff14d7aca45419054434365802dbf, user_id=8ae25b465a47404a80b7f40580e70884, role_id=e3235eda7c654e2abf96ed6a494f5152)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:36:32.086327 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f771267b-aa52-48d8-b657-8063528caba1 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:36:32.102850 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1455/2924] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:36:32 2022] PUT /identity/v3/projects/8c2ff14d7aca45419054434365802dbf/users/8ae25b465a47404a80b7f40580e70884/roles/e3235eda7c654e2abf96ed6a494f5152 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:36:32.108507 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fbfe29-646b-4a99-b870-b6934eeb2e3e None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:32.108793 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fbfe29-646b-4a99-b870-b6934eeb2e3e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:32.109062 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fbfe29-646b-4a99-b870-b6934eeb2e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:32.128879 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-c2fbfe29-646b-4a99-b870-b6934eeb2e3e None None] MFA Rules not processed for user `c96b7a6c0bc940caab4581734c1919ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:36:32.130169 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c2fbfe29-646b-4a99-b870-b6934eeb2e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:36:32.189512 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1470/2925] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:36:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:36:32.199273 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f771267b-aa52-48d8-b657-8063528caba1 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:36:32.201154 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-f771267b-aa52-48d8-b657-8063528caba1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:36:32.258452 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f771267b-aa52-48d8-b657-8063528caba1 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:36:32.262522 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a9296-dba4-4a21-b1c1-0513f5bd0736 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:36:32.263120 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a9296-dba4-4a21-b1c1-0513f5bd0736 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:32.263397 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a9296-dba4-4a21-b1c1-0513f5bd0736 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:32.263667 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a9296-dba4-4a21-b1c1-0513f5bd0736 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:32.269670 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:36:32.273805 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397a9296-dba4-4a21-b1c1-0513f5bd0736 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:36:32.276083 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397a9296-dba4-4a21-b1c1-0513f5bd0736 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:36:32.286577 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1456/2926] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:36:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:36:36.291455 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26cd4fa-e999-43c6-98a9-eaefa648f55c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:36.291931 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26cd4fa-e999-43c6-98a9-eaefa648f55c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:36.292034 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26cd4fa-e999-43c6-98a9-eaefa648f55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:36.323156 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d26cd4fa-e999-43c6-98a9-eaefa648f55c None None] MFA Rules not processed for user `8ae25b465a47404a80b7f40580e70884`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:36:36.324378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d26cd4fa-e999-43c6-98a9-eaefa648f55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:36:36.391476 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1471/2927] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:36:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:36:36.396863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a9296-dba4-4a21-b1c1-0513f5bd0736 service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:36:36.409167 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a9296-dba4-4a21-b1c1-0513f5bd0736 service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:36:36.414182 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9292ed69-0c5a-4391-8edc-739043de3d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:36:36.414794 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9292ed69-0c5a-4391-8edc-739043de3d52 admin admin] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:36.415070 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9292ed69-0c5a-4391-8edc-739043de3d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:36.415335 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9292ed69-0c5a-4391-8edc-739043de3d52 admin admin] PATH_INFO: `/v3/projects` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:36.415971 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9292ed69-0c5a-4391-8edc-739043de3d52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:36:36.417909 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9292ed69-0c5a-4391-8edc-739043de3d52 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:36:36.447472 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1457/2928] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Aug 31 05:36:36 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 05:36:36.451673 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26cd4fa-e999-43c6-98a9-eaefa648f55c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:36:36.462638 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26cd4fa-e999-43c6-98a9-eaefa648f55c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:36:36.523074 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4e2a88-6ad2-4450-8bc7-032cda0f7886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:36:36.523695 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e2a88-6ad2-4450-8bc7-032cda0f7886 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:36.523978 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e2a88-6ad2-4450-8bc7-032cda0f7886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:36.524243 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e2a88-6ad2-4450-8bc7-032cda0f7886 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:36.524896 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4e2a88-6ad2-4450-8bc7-032cda0f7886 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:36:36.526900 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4e2a88-6ad2-4450-8bc7-032cda0f7886 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:36:36.544950 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:36:36.553701 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1472/2929] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Aug 31 05:36:36 2022] POST /identity/v3/users => generated 296 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 05:36:36.554494 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9292ed69-0c5a-4391-8edc-739043de3d52 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:36:36.565568 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9292ed69-0c5a-4391-8edc-739043de3d52 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:36:36.570455 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75dd5ec-995f-432c-9179-01e8b10ac622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:36:36.571067 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75dd5ec-995f-432c-9179-01e8b10ac622 admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:36.571343 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75dd5ec-995f-432c-9179-01e8b10ac622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:36.571640 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75dd5ec-995f-432c-9179-01e8b10ac622 admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:36.572296 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75dd5ec-995f-432c-9179-01e8b10ac622 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:36:36.573936 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75dd5ec-995f-432c-9179-01e8b10ac622 admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:36:36.585423 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1458/2930] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 05:36:36.589613 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4e2a88-6ad2-4450-8bc7-032cda0f7886 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:36:36.599939 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4e2a88-6ad2-4450-8bc7-032cda0f7886 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:36:36.605133 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be5cac-7334-42b2-be5b-8a4aeb3d6978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:36:36.606259 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be5cac-7334-42b2-be5b-8a4aeb3d6978 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:36.606735 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be5cac-7334-42b2-be5b-8a4aeb3d6978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:36.607010 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be5cac-7334-42b2-be5b-8a4aeb3d6978 admin admin] PATH_INFO: `/v3/projects/3be183c58657482f83b0040e73f93f3d/users/4b33a4f0d9834e5bbb77b7ea7443f477/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:36.623031 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7be5cac-7334-42b2-be5b-8a4aeb3d6978 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3be183c58657482f83b0040e73f93f3d, user_id=4b33a4f0d9834e5bbb77b7ea7443f477, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:36:36.625722 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7be5cac-7334-42b2-be5b-8a4aeb3d6978 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:36:36.640306 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1473/2931] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:36:36 2022] PUT /identity/v3/projects/3be183c58657482f83b0040e73f93f3d/users/4b33a4f0d9834e5bbb77b7ea7443f477/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:36:36.644926 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75dd5ec-995f-432c-9179-01e8b10ac622 admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:36:36.656024 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75dd5ec-995f-432c-9179-01e8b10ac622 admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:36:36.715803 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5ca2ae-2796-4bc7-afab-d743163b517b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:36:36.716417 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ca2ae-2796-4bc7-afab-d743163b517b admin admin] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:36.716695 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ca2ae-2796-4bc7-afab-d743163b517b admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:36.716965 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ca2ae-2796-4bc7-afab-d743163b517b admin admin] PATH_INFO: `/v3/roles` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:36.717591 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5ca2ae-2796-4bc7-afab-d743163b517b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:36:36.719240 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5ca2ae-2796-4bc7-afab-d743163b517b admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:36:36.729920 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1459/2932] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Aug 31 05:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 05:36:36.734414 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be5cac-7334-42b2-be5b-8a4aeb3d6978 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:36:36.745296 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be5cac-7334-42b2-be5b-8a4aeb3d6978 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:36:36.751153 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea72fb-2cee-4fa8-9573-d021beab80bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:36:36.751790 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea72fb-2cee-4fa8-9573-d021beab80bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:36.752061 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea72fb-2cee-4fa8-9573-d021beab80bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:36.752320 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea72fb-2cee-4fa8-9573-d021beab80bb admin admin] PATH_INFO: `/v3/projects/3be183c58657482f83b0040e73f93f3d/users/4b33a4f0d9834e5bbb77b7ea7443f477/roles/c3d5589dd2864be9a0f26f5802d874cd` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:36.754516 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cea72fb-2cee-4fa8-9573-d021beab80bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3be183c58657482f83b0040e73f93f3d, user_id=4b33a4f0d9834e5bbb77b7ea7443f477, role_id=c3d5589dd2864be9a0f26f5802d874cd)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:36:36.757161 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cea72fb-2cee-4fa8-9573-d021beab80bb admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:36:36.767814 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1474/2933] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Aug 31 05:36:36 2022] PUT /identity/v3/projects/3be183c58657482f83b0040e73f93f3d/users/4b33a4f0d9834e5bbb77b7ea7443f477/roles/c3d5589dd2864be9a0f26f5802d874cd => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:36:41.183666 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec8c11-7eaf-4e43-b64d-75d7e03c2501 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:41.185855 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec8c11-7eaf-4e43-b64d-75d7e03c2501 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:41.185855 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec8c11-7eaf-4e43-b64d-75d7e03c2501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:41.227260 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-9dec8c11-7eaf-4e43-b64d-75d7e03c2501 None None] MFA Rules not processed for user `4b33a4f0d9834e5bbb77b7ea7443f477`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:36:41.228475 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-9dec8c11-7eaf-4e43-b64d-75d7e03c2501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:36:41.302298 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1460/2934] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:36:41 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 05:36:41.318517 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea72fb-2cee-4fa8-9573-d021beab80bb admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:36:41.320909 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1cea72fb-2cee-4fa8-9573-d021beab80bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:36:41.412030 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea72fb-2cee-4fa8-9573-d021beab80bb admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:36:41.418107 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5a53da-9f8f-418c-9253-f304732e842b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:36:41.418951 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a53da-9f8f-418c-9253-f304732e842b service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:36:41.419301 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a53da-9f8f-418c-9253-f304732e842b service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:36:41.419639 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a53da-9f8f-418c-9253-f304732e842b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:36:41.487587 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5a53da-9f8f-418c-9253-f304732e842b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:36:41.489266 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5a53da-9f8f-418c-9253-f304732e842b service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:36:41.500294 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1475/2935] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:36:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:36:51.299500 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=80,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:36:51.494474 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=71,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:37:02.364046 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079940ff-eb82-4c0b-ac53-d46c4e228510 None None] REQUEST_METHOD: `POST` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:37:02.364551 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079940ff-eb82-4c0b-ac53-d46c4e228510 None None] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:37:02.364551 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079940ff-eb82-4c0b-ac53-d46c4e228510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:37:02.394996 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.auth.core [None req-079940ff-eb82-4c0b-ac53-d46c4e228510 None None] MFA Rules not processed for user `8ae25b465a47404a80b7f40580e70884`. Rule list: `[]` (Enabled: `True`). {{(pid=81601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:37:02.396207 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-079940ff-eb82-4c0b-ac53-d46c4e228510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:37:02.422958 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1461/2936] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:37:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:37:02.774944 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5a53da-9f8f-418c-9253-f304732e842b service nova] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:37:02.869794 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5a53da-9f8f-418c-9253-f304732e842b service nova] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:37:02.874578 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752b0896-6e50-43f9-8ca4-a875bce09254 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:37:02.875180 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b0896-6e50-43f9-8ca4-a875bce09254 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:37:02.875453 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b0896-6e50-43f9-8ca4-a875bce09254 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:37:02.875718 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b0896-6e50-43f9-8ca4-a875bce09254 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:37:02.877862 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-752b0896-6e50-43f9-8ca4-a875bce09254 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:37:02.888560 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752b0896-6e50-43f9-8ca4-a875bce09254 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:37:02.890037 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752b0896-6e50-43f9-8ca4-a875bce09254 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:37:02.898157 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1476/2937] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:37:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:37:12.420863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=15 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:37:12.894467 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=35 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:37:19.105860 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079940ff-eb82-4c0b-ac53-d46c4e228510 None None] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:37:19.127333 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079940ff-eb82-4c0b-ac53-d46c4e228510 None None] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:37:19.131732 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19804cd3-070a-49dd-b63b-89d184479959 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:37:19.132324 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19804cd3-070a-49dd-b63b-89d184479959 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:37:19.132598 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19804cd3-070a-49dd-b63b-89d184479959 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:37:19.132864 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19804cd3-070a-49dd-b63b-89d184479959 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:37:19.152777 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19804cd3-070a-49dd-b63b-89d184479959 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:37:19.154287 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19804cd3-070a-49dd-b63b-89d184479959 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:37:19.163888 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1462/2938] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:37:19 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:37:29.159431 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=13 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:37:55.731380 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752b0896-6e50-43f9-8ca4-a875bce09254 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:37:55.733054 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-752b0896-6e50-43f9-8ca4-a875bce09254 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:37:55.823782 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752b0896-6e50-43f9-8ca4-a875bce09254 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:37:55.829251 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a48195-7c5c-4525-bb19-15aa1009937b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd4387035099416789009f0ecb8ad282', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:37:55.829850 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a48195-7c5c-4525-bb19-15aa1009937b service placement] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:37:55.830159 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a48195-7c5c-4525-bb19-15aa1009937b service placement] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:37:55.830419 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a48195-7c5c-4525-bb19-15aa1009937b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:37:55.832309 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a7a48195-7c5c-4525-bb19-15aa1009937b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:37:55.843724 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a48195-7c5c-4525-bb19-15aa1009937b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:37:55.845291 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a48195-7c5c-4525-bb19-15aa1009937b service placement] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:37:55.855557 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1477/2939] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:37:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:38:05.851207 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=41 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:39:36.248792 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19804cd3-070a-49dd-b63b-89d184479959 service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:39:36.258970 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19804cd3-070a-49dd-b63b-89d184479959 service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:39:36.264006 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f36cf-c050-4086-bbe9-a225ab9b59be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:39:36.264605 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f36cf-c050-4086-bbe9-a225ab9b59be service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:39:36.264874 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f36cf-c050-4086-bbe9-a225ab9b59be service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:39:36.265137 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f36cf-c050-4086-bbe9-a225ab9b59be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:39:36.267551 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-cc3f36cf-c050-4086-bbe9-a225ab9b59be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:39:36.307345 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3f36cf-c050-4086-bbe9-a225ab9b59be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:39:36.308878 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3f36cf-c050-4086-bbe9-a225ab9b59be service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:39:36.340668 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1463/2940] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:39:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:39:39.761496 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38197e-1452-4cac-adb8-9aa269f451e0 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:39:39.762059 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38197e-1452-4cac-adb8-9aa269f451e0 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:39:39.762059 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38197e-1452-4cac-adb8-9aa269f451e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:39:39.781384 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ea38197e-1452-4cac-adb8-9aa269f451e0 None None] MFA Rules not processed for user `8ae25b465a47404a80b7f40580e70884`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:39:39.782612 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ea38197e-1452-4cac-adb8-9aa269f451e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:39:39.789653 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1478/2941] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:39:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:39:45.522919 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f36cf-c050-4086-bbe9-a225ab9b59be service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:39:45.533188 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f36cf-c050-4086-bbe9-a225ab9b59be service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:39:45.540205 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756e35eb-172d-4201-bbe1-d2021683dc1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:39:45.541173 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e35eb-172d-4201-bbe1-d2021683dc1a service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:39:45.541631 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e35eb-172d-4201-bbe1-d2021683dc1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:39:45.541988 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e35eb-172d-4201-bbe1-d2021683dc1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:39:45.544772 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-756e35eb-172d-4201-bbe1-d2021683dc1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:39:45.559581 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756e35eb-172d-4201-bbe1-d2021683dc1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:39:45.561018 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756e35eb-172d-4201-bbe1-d2021683dc1a service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:39:45.572518 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1464/2942] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:39:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:39:49.776069 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:39:55.567433 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=24 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:42:00.683991 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38197e-1452-4cac-adb8-9aa269f451e0 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:42:00.686394 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ea38197e-1452-4cac-adb8-9aa269f451e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:42:00.778830 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38197e-1452-4cac-adb8-9aa269f451e0 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:42:00.782800 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb499366-7c52-47ab-9690-3635997cb228 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8300075371341a6b9cd4dcf8ec7fa8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:42:00.783408 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb499366-7c52-47ab-9690-3635997cb228 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:42:00.783682 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb499366-7c52-47ab-9690-3635997cb228 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:42:00.783955 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb499366-7c52-47ab-9690-3635997cb228 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:42:00.792977 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb499366-7c52-47ab-9690-3635997cb228 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:42:00.794492 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb499366-7c52-47ab-9690-3635997cb228 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:42:00.803580 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1479/2943] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:42:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:42:07.221785 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756e35eb-172d-4201-bbe1-d2021683dc1a service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:42:07.231570 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756e35eb-172d-4201-bbe1-d2021683dc1a service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:42:07.236112 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ddb90f-7645-48b2-8d9d-5b40e656121e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:42:07.236704 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddb90f-7645-48b2-8d9d-5b40e656121e service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:42:07.236977 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddb90f-7645-48b2-8d9d-5b40e656121e service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:42:07.237309 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddb90f-7645-48b2-8d9d-5b40e656121e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:42:07.239246 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-b7ddb90f-7645-48b2-8d9d-5b40e656121e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:42:07.262978 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ddb90f-7645-48b2-8d9d-5b40e656121e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:42:07.264395 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ddb90f-7645-48b2-8d9d-5b40e656121e service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:42:07.273277 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1465/2944] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:42:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:42:10.799367 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=35 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:42:11.916725 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2aeb6-c904-41d0-b187-d4c531c7c375 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:42:11.917371 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2aeb6-c904-41d0-b187-d4c531c7c375 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:42:11.917371 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2aeb6-c904-41d0-b187-d4c531c7c375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:42:11.939013 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-03e2aeb6-c904-41d0-b187-d4c531c7c375 None None] MFA Rules not processed for user `8ae25b465a47404a80b7f40580e70884`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 05:42:11.940294 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-03e2aeb6-c904-41d0-b187-d4c531c7c375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:42:11.952431 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1480/2945] 10.222.0.36 () {58 vars in 958 bytes} [Wed Aug 31 05:42:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 05:42:12.291211 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ddb90f-7645-48b2-8d9d-5b40e656121e service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:42:12.300420 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ddb90f-7645-48b2-8d9d-5b40e656121e service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:42:12.305070 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001552e7-6b54-49ed-8a82-180fc763e101 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:42:12.305670 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001552e7-6b54-49ed-8a82-180fc763e101 service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:42:12.305949 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001552e7-6b54-49ed-8a82-180fc763e101 service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:42:12.306253 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001552e7-6b54-49ed-8a82-180fc763e101 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:42:12.308263 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-001552e7-6b54-49ed-8a82-180fc763e101 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:42:12.318633 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001552e7-6b54-49ed-8a82-180fc763e101 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:42:12.320083 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001552e7-6b54-49ed-8a82-180fc763e101 service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:42:12.329489 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1466/2946] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:42:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:42:12.812471 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e2aeb6-c904-41d0-b187-d4c531c7c375 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:42:12.824977 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e2aeb6-c904-41d0-b187-d4c531c7c375 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:42:12.830640 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca7b364-86e4-4628-a3f6-ad0025b771a4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd4387035099416789009f0ecb8ad282', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:42:12.831270 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7b364-86e4-4628-a3f6-ad0025b771a4 service placement] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:42:12.831545 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7b364-86e4-4628-a3f6-ad0025b771a4 service placement] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:42:12.831815 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7b364-86e4-4628-a3f6-ad0025b771a4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:42:12.833535 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8ca7b364-86e4-4628-a3f6-ad0025b771a4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:42:12.845429 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca7b364-86e4-4628-a3f6-ad0025b771a4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:42:12.847054 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca7b364-86e4-4628-a3f6-ad0025b771a4 service placement] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:42:12.857733 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1481/2947] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:42:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:42:15.490472 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001552e7-6b54-49ed-8a82-180fc763e101 service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:42:15.505870 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001552e7-6b54-49ed-8a82-180fc763e101 service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:42:15.512442 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc1545-b83f-4417-9e48-d93f847f440a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:42:15.513059 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc1545-b83f-4417-9e48-d93f847f440a service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:42:15.513341 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc1545-b83f-4417-9e48-d93f847f440a service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:42:15.513609 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc1545-b83f-4417-9e48-d93f847f440a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:42:15.515591 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-e2cc1545-b83f-4417-9e48-d93f847f440a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:42:15.529249 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cc1545-b83f-4417-9e48-d93f847f440a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:42:15.530900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cc1545-b83f-4417-9e48-d93f847f440a service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:42:15.542385 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1467/2948] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:42:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:42:22.852991 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=9 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:42:25.537630 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=23 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:42:26.031092 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca7b364-86e4-4628-a3f6-ad0025b771a4 service placement] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:42:26.044907 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca7b364-86e4-4628-a3f6-ad0025b771a4 service placement] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:42:26.050229 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6757aed-b557-4636-b813-492e7bdb0016 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:42:26.050835 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6757aed-b557-4636-b813-492e7bdb0016 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:42:26.051103 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6757aed-b557-4636-b813-492e7bdb0016 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:42:26.051365 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6757aed-b557-4636-b813-492e7bdb0016 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:42:26.053120 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e6757aed-b557-4636-b813-492e7bdb0016 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:42:26.065378 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6757aed-b557-4636-b813-492e7bdb0016 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:42:26.066982 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6757aed-b557-4636-b813-492e7bdb0016 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:42:26.077416 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1482/2949] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:42:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:42:28.506455 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc1545-b83f-4417-9e48-d93f847f440a service nova] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:42:28.516649 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc1545-b83f-4417-9e48-d93f847f440a service nova] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:42:28.521025 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a803f18-656a-49dd-83f8-506b40fa2c2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:42:28.521613 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a803f18-656a-49dd-83f8-506b40fa2c2a service neutron] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:42:28.521877 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a803f18-656a-49dd-83f8-506b40fa2c2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:42:28.522182 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a803f18-656a-49dd-83f8-506b40fa2c2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:42:28.531518 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a803f18-656a-49dd-83f8-506b40fa2c2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:42:28.533002 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a803f18-656a-49dd-83f8-506b40fa2c2a service neutron] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:42:28.543025 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1468/2950] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:42:28 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:42:36.073215 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=6 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:42:38.538770 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=6 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:43:00.557564 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6757aed-b557-4636-b813-492e7bdb0016 service nova] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:43:00.559274 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e6757aed-b557-4636-b813-492e7bdb0016 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:43:00.574658 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6757aed-b557-4636-b813-492e7bdb0016 service nova] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:43:00.581018 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a86de4e-6d87-4d57-91c7-ba6dc4597e02 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd4387035099416789009f0ecb8ad282', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:43:00.581624 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86de4e-6d87-4d57-91c7-ba6dc4597e02 service placement] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:43:00.581934 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86de4e-6d87-4d57-91c7-ba6dc4597e02 service placement] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:43:00.582256 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86de4e-6d87-4d57-91c7-ba6dc4597e02 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:43:00.591185 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a86de4e-6d87-4d57-91c7-ba6dc4597e02 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:43:00.592771 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a86de4e-6d87-4d57-91c7-ba6dc4597e02 service placement] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:43:00.602825 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1483/2951] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:43:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:43:10.598577 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=6 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:44:27.340995 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a803f18-656a-49dd-83f8-506b40fa2c2a service neutron] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:44:27.355690 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a803f18-656a-49dd-83f8-506b40fa2c2a service neutron] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:44:27.362412 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700719ec-8e37-4d1f-bc1e-82f35a8f08e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27665b1574a4db7b7a9771f7d775826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:44:27.363323 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700719ec-8e37-4d1f-bc1e-82f35a8f08e6 service nova] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:44:27.363817 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700719ec-8e37-4d1f-bc1e-82f35a8f08e6 service nova] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:44:27.364360 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700719ec-8e37-4d1f-bc1e-82f35a8f08e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:44:27.366381 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-700719ec-8e37-4d1f-bc1e-82f35a8f08e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:44:27.380046 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700719ec-8e37-4d1f-bc1e-82f35a8f08e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:44:27.381746 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700719ec-8e37-4d1f-bc1e-82f35a8f08e6 service nova] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:44:27.395398 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1469/2952] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:44:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:44:32.909444 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a86de4e-6d87-4d57-91c7-ba6dc4597e02 service placement] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:44:32.911037 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1a86de4e-6d87-4d57-91c7-ba6dc4597e02 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:44:32.924724 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a86de4e-6d87-4d57-91c7-ba6dc4597e02 service placement] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:44:32.928369 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41892917-236c-4deb-99aa-1aa35baa7dd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '090e1409506542f6831e1f65d44aae74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:44:32.928970 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41892917-236c-4deb-99aa-1aa35baa7dd5 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:44:32.929246 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41892917-236c-4deb-99aa-1aa35baa7dd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:44:32.929514 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41892917-236c-4deb-99aa-1aa35baa7dd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:44:33.004225 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41892917-236c-4deb-99aa-1aa35baa7dd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:44:33.005587 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41892917-236c-4deb-99aa-1aa35baa7dd5 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:44:33.024655 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1484/2953] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:44:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:44:36.167764 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700719ec-8e37-4d1f-bc1e-82f35a8f08e6 service nova] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:44:36.180626 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700719ec-8e37-4d1f-bc1e-82f35a8f08e6 service nova] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:44:36.186248 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da7c5eb-bb62-4687-8ec7-69c02f36f366 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd4387035099416789009f0ecb8ad282', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:44:36.186877 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da7c5eb-bb62-4687-8ec7-69c02f36f366 service placement] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:44:36.187150 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da7c5eb-bb62-4687-8ec7-69c02f36f366 service placement] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:44:36.187395 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da7c5eb-bb62-4687-8ec7-69c02f36f366 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:44:36.189227 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-2da7c5eb-bb62-4687-8ec7-69c02f36f366 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:44:36.202072 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da7c5eb-bb62-4687-8ec7-69c02f36f366 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:44:36.203863 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da7c5eb-bb62-4687-8ec7-69c02f36f366 service placement] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:44:36.215365 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1470/2954] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:44:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:44:39.328536 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41892917-236c-4deb-99aa-1aa35baa7dd5 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:44:39.354297 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41892917-236c-4deb-99aa-1aa35baa7dd5 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:44:39.362354 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6818e1a6-23f1-4416-91b7-638b6176614d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd4387035099416789009f0ecb8ad282', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:44:39.362808 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818e1a6-23f1-4416-91b7-638b6176614d service placement] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:44:39.363302 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818e1a6-23f1-4416-91b7-638b6176614d service placement] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:44:39.363728 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818e1a6-23f1-4416-91b7-638b6176614d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:44:39.366310 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6818e1a6-23f1-4416-91b7-638b6176614d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:44:39.391031 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6818e1a6-23f1-4416-91b7-638b6176614d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:44:39.392674 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6818e1a6-23f1-4416-91b7-638b6176614d service placement] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:44:39.405533 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da7c5eb-bb62-4687-8ec7-69c02f36f366 service placement] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:44:39.419139 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1485/2955] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:44:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:44:39.430144 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da7c5eb-bb62-4687-8ec7-69c02f36f366 service placement] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:44:39.439450 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33d411-6f8a-44fd-8e1a-c2cd299e2c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:44:39.439858 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33d411-6f8a-44fd-8e1a-c2cd299e2c8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:44:39.440200 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33d411-6f8a-44fd-8e1a-c2cd299e2c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:44:39.440528 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33d411-6f8a-44fd-8e1a-c2cd299e2c8e admin admin] PATH_INFO: `/v3/users/8ae25b465a47404a80b7f40580e70884` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:44:39.443180 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a33d411-6f8a-44fd-8e1a-c2cd299e2c8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ae25b465a47404a80b7f40580e70884)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:44:39.445378 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a33d411-6f8a-44fd-8e1a-c2cd299e2c8e admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:44:39.500676 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-1a33d411-6f8a-44fd-8e1a-c2cd299e2c8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ae25b465a47404a80b7f40580e70884'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:44:39.507408 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-1a33d411-6f8a-44fd-8e1a-c2cd299e2c8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ae25b465a47404a80b7f40580e70884'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:44:39.508080 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-1a33d411-6f8a-44fd-8e1a-c2cd299e2c8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ae25b465a47404a80b7f40580e70884'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:44:39.515294 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-1a33d411-6f8a-44fd-8e1a-c2cd299e2c8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ae25b465a47404a80b7f40580e70884'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:44:39.515907 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-1a33d411-6f8a-44fd-8e1a-c2cd299e2c8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ae25b465a47404a80b7f40580e70884'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:44:39.533069 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1471/2956] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:44:39 2022] DELETE /identity/v3/users/8ae25b465a47404a80b7f40580e70884 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:44:39.710883 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6818e1a6-23f1-4416-91b7-638b6176614d service placement] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:44:39.722975 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6818e1a6-23f1-4416-91b7-638b6176614d service placement] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:44:39.789206 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e745dd3d-4083-4b9c-93b9-0a4e286ee0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:44:39.789878 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745dd3d-4083-4b9c-93b9-0a4e286ee0a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:44:39.790191 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745dd3d-4083-4b9c-93b9-0a4e286ee0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:44:39.790456 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745dd3d-4083-4b9c-93b9-0a4e286ee0a5 admin admin] PATH_INFO: `/v3/projects/8c2ff14d7aca45419054434365802dbf` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:44:39.791574 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e745dd3d-4083-4b9c-93b9-0a4e286ee0a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c2ff14d7aca45419054434365802dbf)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:44:39.793517 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e745dd3d-4083-4b9c-93b9-0a4e286ee0a5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:44:39.842187 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e745dd3d-4083-4b9c-93b9-0a4e286ee0a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c2ff14d7aca45419054434365802dbf', 'request_id': 'req-e745dd3d-4083-4b9c-93b9-0a4e286ee0a5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:44:39.849292 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e745dd3d-4083-4b9c-93b9-0a4e286ee0a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c2ff14d7aca45419054434365802dbf', 'request_id': 'req-e745dd3d-4083-4b9c-93b9-0a4e286ee0a5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:44:39.858742 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e745dd3d-4083-4b9c-93b9-0a4e286ee0a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c2ff14d7aca45419054434365802dbf', 'request_id': 'req-e745dd3d-4083-4b9c-93b9-0a4e286ee0a5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:44:39.863079 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:44:39.883056 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e745dd3d-4083-4b9c-93b9-0a4e286ee0a5 admin admin] The token cache is being invalidate because project 8c2ff14d7aca45419054434365802dbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:44:39.883340 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e745dd3d-4083-4b9c-93b9-0a4e286ee0a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:44:39.883948 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e745dd3d-4083-4b9c-93b9-0a4e286ee0a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:44:39.885441 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1486/2957] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:44:39 2022] DELETE /identity/v3/projects/8c2ff14d7aca45419054434365802dbf => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:44:39.889509 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33d411-6f8a-44fd-8e1a-c2cd299e2c8e admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:44:39.891022 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-1a33d411-6f8a-44fd-8e1a-c2cd299e2c8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:44:39.904662 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33d411-6f8a-44fd-8e1a-c2cd299e2c8e admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:44:39.963061 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c200ba97-334b-4a86-8e2a-187c9ffe97db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:44:39.963698 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200ba97-334b-4a86-8e2a-187c9ffe97db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:44:39.963972 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200ba97-334b-4a86-8e2a-187c9ffe97db admin admin] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:44:39.964236 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200ba97-334b-4a86-8e2a-187c9ffe97db admin admin] PATH_INFO: `/v3/users/4b33a4f0d9834e5bbb77b7ea7443f477` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:44:39.965518 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c200ba97-334b-4a86-8e2a-187c9ffe97db admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b33a4f0d9834e5bbb77b7ea7443f477)` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:44:39.967603 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c200ba97-334b-4a86-8e2a-187c9ffe97db admin admin] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:44:40.017334 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c200ba97-334b-4a86-8e2a-187c9ffe97db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b33a4f0d9834e5bbb77b7ea7443f477'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:44:40.018547 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:44:40.026412 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c200ba97-334b-4a86-8e2a-187c9ffe97db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b33a4f0d9834e5bbb77b7ea7443f477'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:44:40.027192 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c200ba97-334b-4a86-8e2a-187c9ffe97db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b33a4f0d9834e5bbb77b7ea7443f477'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:44:40.036007 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c200ba97-334b-4a86-8e2a-187c9ffe97db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b33a4f0d9834e5bbb77b7ea7443f477'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:44:40.036694 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.notifications [None req-c200ba97-334b-4a86-8e2a-187c9ffe97db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b33a4f0d9834e5bbb77b7ea7443f477'} {{(pid=81601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:44:40.053823 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1472/2958] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Aug 31 05:44:39 2022] DELETE /identity/v3/users/4b33a4f0d9834e5bbb77b7ea7443f477 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 05:44:40.239775 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e745dd3d-4083-4b9c-93b9-0a4e286ee0a5 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:44:40.241423 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e745dd3d-4083-4b9c-93b9-0a4e286ee0a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:44:40.256084 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e745dd3d-4083-4b9c-93b9-0a4e286ee0a5 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:44:40.311066 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d242c1-4760-4bbd-abad-2aac9054cc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c96b7a6c0bc940caab4581734c1919ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e3bdb87df7f4087806a21456cb45a01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:44:40.311697 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d242c1-4760-4bbd-abad-2aac9054cc7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:44:40.311962 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d242c1-4760-4bbd-abad-2aac9054cc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:44:40.312223 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d242c1-4760-4bbd-abad-2aac9054cc7a admin admin] PATH_INFO: `/v3/projects/3be183c58657482f83b0040e73f93f3d` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:44:40.313422 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d242c1-4760-4bbd-abad-2aac9054cc7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3be183c58657482f83b0040e73f93f3d)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:44:40.315363 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d242c1-4760-4bbd-abad-2aac9054cc7a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:44:40.362366 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-03d242c1-4760-4bbd-abad-2aac9054cc7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3be183c58657482f83b0040e73f93f3d', 'request_id': 'req-03d242c1-4760-4bbd-abad-2aac9054cc7a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:44:40.368780 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-03d242c1-4760-4bbd-abad-2aac9054cc7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3be183c58657482f83b0040e73f93f3d', 'request_id': 'req-03d242c1-4760-4bbd-abad-2aac9054cc7a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:44:40.377387 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-03d242c1-4760-4bbd-abad-2aac9054cc7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3be183c58657482f83b0040e73f93f3d', 'request_id': 'req-03d242c1-4760-4bbd-abad-2aac9054cc7a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:44:40.388272 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-03d242c1-4760-4bbd-abad-2aac9054cc7a admin admin] The token cache is being invalidate because project 3be183c58657482f83b0040e73f93f3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 05:44:40.388559 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-03d242c1-4760-4bbd-abad-2aac9054cc7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:44:40.389207 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-03d242c1-4760-4bbd-abad-2aac9054cc7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 05:44:40.390769 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1487/2959] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Aug 31 05:44:40 2022] DELETE /identity/v3/projects/3be183c58657482f83b0040e73f93f3d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 05:44:50.050710 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=12,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:44:50.385518 dv-cinder-852208-14-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:DELETE=8,keystone:SELECT=58,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 05:48:03.625039 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c200ba97-334b-4a86-8e2a-187c9ffe97db admin admin] Authenticating user token {{(pid=81601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 05:48:03.626779 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-c200ba97-334b-4a86-8e2a-187c9ffe97db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:48:03.771959 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c200ba97-334b-4a86-8e2a-187c9ffe97db admin admin] Validating token access rules against request {{(pid=81601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 05:48:03.833203 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eca77a6-0a85-4107-a7e5-5e75c1164a66 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd4387035099416789009f0ecb8ad282', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e01ce1b1282e44a39d659b87f120995e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 05:48:03.833799 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca77a6-0a85-4107-a7e5-5e75c1164a66 service placement] REQUEST_METHOD: `GET` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 05:48:03.834086 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca77a6-0a85-4107-a7e5-5e75c1164a66 service placement] SCRIPT_NAME: `/identity` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 05:48:03.834362 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca77a6-0a85-4107-a7e5-5e75c1164a66 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 05:48:03.836337 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.fernet_utils [None req-4eca77a6-0a85-4107-a7e5-5e75c1164a66 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 05:48:03.847936 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eca77a6-0a85-4107-a7e5-5e75c1164a66 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 05:48:03.849920 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eca77a6-0a85-4107-a7e5-5e75c1164a66 service placement] RBAC: Authorization granted {{(pid=81601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 05:48:03.859900 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: [pid: 81601|app: 0|req: 1473/2960] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Aug 31 05:48:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 05:48:13.855736 dv-cinder-852208-14-iscsi devstack@keystone.service[81601]: DEBUG dbcounter [-] [81601] Writing DB stats keystone:SELECT=89 {{(pid=81601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}